Applied NLP Engineer - Dataset and Model Curation

About the Role

As a Data Scientist / Applied NLP Engineer, you’ll play a central role in improving the quality of our NLP and LLM systems by building the datasets and evaluation processes that drive model performance. You’ll curate and label large-scale text datasets, define annotation policies, benchmark models across releases, analyze failure patterns, and develop automated LLM-powered labeling and QA pipelines. This role combines hands-on NLP, data engineering, and applied machine learning, requiring strong analytical thinking, exceptional attention to detail, and the ability to make sound decisions in ambiguous edge cases. If you enjoy solving complex language problems, improving models through better data, and turning experimentation into scalable production workflows, you’ll thrive on our team.

You Have:

• Excellent English (IELTS 7.5+ or equivalent) with hands-on dataset labeling and annotation quality control: sample labeling will require deep contextual understanding of text (subtle intent, boundaries, and edge cases), plus clear writing of labeling policies and QA notes
• Strong NLP background (ideally sequence labeling/NER/token-level tasks) with experience QA-ing ML models: error analysis, failure clustering, precision/recall trade-offs
• Comfort with large-scale text datasets and dataset selection / curation: sampling, filtering, aggregation, scalable human-review workflows, choosing sources, and selecting high-value samples under coverage and diversity constraints
• Experience with data post-processing after selection or agentic labeling (validation, cleanup, format conversion, quality gates) and ability to define and revise labeling policies so rules stay consistent and enforceable across edge cases
• Experience with model benchmarking (comparative results across checkpoints/systems) and practical prompt engineering for structured extraction / annotation tasks
• Experience deploying or operationalizing LLMs in automated pipelines (API integration, batch jobs, retries, throughput/cost awareness) and solid Python for data work and scripting (able to read and extend existing analysis code)
• High attention to detail, strong judgment under ambiguity, and clear written communication of decisions and trade-offs

What You'll Do:

• Model QA & benchmarking: systematically evaluate entity predictions, identify error patterns, and distinguish model failures from label/data issues; build and maintain evaluation benchmarks, run comparative evaluations across versions, track key metrics, and communicate performance gains and regressions
• Dataset curation & labeling: curate labeled datasets at scale: process, filter, audit, and analyze large text corpora and span-labeled data; evaluate candidate sources, design selection strategies for diverse/entity-rich training and eval sets, and create/review/correct entity annotations with careful judgment on ambiguous cases
• Annotation systems & data readiness: write and tighten labeling policies (entity definitions, span boundaries, inconsistencies, gaps, edge cases); design and iterate agentic systems for labeling and label fixing; clean, normalize, validate, and convert selected/labeled outputs into training-ready datasets
• LLM deployment & automation: wire models into production-like agentic labeling/QA workflows; automate large-scale API runs, retries, batching, and result collection

Nice to Have:

• Multilingual and large-corpus NLP experience: experience working with multilingual or cross-locale text datasets, including language-specific preprocessing, normalization, and the use of embeddings, similarity search, clustering, or retrieval methods to explore, organize, and select high-value samples from large corpora
• End-to-end model development experience: hands-on experience training, fine-tuning, evaluating, and iterating on NLP or LLM-based models, with an understanding of how dataset quality, annotation decisions, and model configuration affect downstream performance
• Annotation schema and dataset lifecycle management: experience designing large annotation taxonomies, maintaining consistent entity definitions and span-boundary rules as requirements evolve, and managing versioned datasets, experiment tracking, model artifacts, and reproducible release workflows
• Production-oriented applied ML judgment: practical experience operating LLM or NLP systems at scale and making informed trade-offs between quality, cost, latency, throughput, and implementation complexity in fast-moving environments with ambiguous requirements and evolving label definitions