Machine Learning Assisted Text Annotation

Figure Eight Federal’s platform seamlessly combines the annotations of our expert human workforce with cutting-edge machine learning predictions to deliver the highest-quality annotated text for your Natural Language Processing (NLP) projects.

Here’s how it works:

Users can upload their plain text directly, or a JSON (Java Script Object Notation) of custom tokens, spans, and text predictions. Users can also easily configure text classes for our annotators using templates with custom class instructions and in-tool help functionality, and even create test questions to evaluate our annotators on comprehension and accuracy. Even more, users can aggregate correct labels from multiple annotators using our inter-annotator agreement score, so they can see exactly where and how the model required human correction.

Label Experience for Machine Learning Assisted Text Annotation

Using our Machine Learning Assisted Text Annotation (MLATA) enhancements, our human annotators quickly and accurately assign class labels to individual text tokens and spans to enable entity extraction and parts-of-speech labeling. Users can also leverage their own tokenizers, or use spaCy, NLTK, or Stanford within our platform to create tokens quickly in dozens of different languages, or use their own NLP model predictions to help our annotators increase precision, recall, and speed. Our Federal and corporate clients have successfully used our powerful NLP capabilities for training chatbots, text search tools, document comprehension, and many other sophisticated use cases.

Our In-Tool Functionality Provides an Easy and Efficient Labeling Experience:

  • Hotkey support for major functionalities allowing annotators to optimize their process
  • In-tool customizable lookup for tokens and spans that aid reading comprehension
  • Label all instances of tokens or spans simultaneously and minimize clicks
  • See annotation context and instructions in-line to quickly view guidance and tips
  • No highlighting required!

Our Quality Control Tools Ensure High-Quality, Low Bias Annotations

  • User-defined Test Questions test for precise class labels and span lengths
  • Inter-annotator agreement scores combine the best output from multiple annotators
  • Utilize machine learning predictions to increase speed and accuracy and create more precise data for retraining

Figure Eight Federal’s Platform Provides Simple Model Validation and Retraining for Your Existing NLP Tools:

Users can upload their NLP predictions for review by our human annotators. Every annotation is then recorded as being “machine” or “human” labeled in order to track class level accuracy, recall, and precision. Users can then reduce annotator bias with test questions that can test for true negatives and false positives from the model or the annotator. Low-performing corrected labels can even be fed back into the NLP model for retraining, to further reduce error rates quickly and cost-effectively.

Figure Eight Federal can make your machine learning projects a success. Contact us and we’ll get you started.