Search Generalists

Role Overview

Mercor is seeking detail-oriented Search Generalist Experts to support a high-impact project with a leading AI research lab. In this role, you will help evaluate and improve how advanced AI systems perform on real-world search and browsing tasks.

This work includes assessing model outputs for factuality, helpfulness, completeness, and judgment quality across a broad range of user queries. You will contribute to structured evaluation workflows that help train, benchmark, and refine frontier AI systems. This is a strong fit for excellent generalists who are sharp researchers, strong writers, and comfortable making nuanced quality judgments at scale.

Key Responsibilities

• Evaluate AI-generated search responses for factual accuracy, helpfulness, clarity, completeness, and overall quality.
• Assess whether models use search appropriately and whether search queries are well-formed and effective.
• Compare model responses side by side and provide concise, defensible rationales.
• Write and refine prompts, golden answers, rubric criteria, and edge cases for search-related evaluations.
• Apply project guidelines consistently across ambiguous, multi-step, and real-world search tasks.
• Identify recurring failure modes and escalate unclear cases or rubric gaps to project leads.
• Participate in calibration, QA, and feedback loops to maintain strong agreement and quality standards.

Qualifications

• Excellent written English and strong online research skills.
• Strong judgment when synthesizing information from multiple sources.
• Ability to distinguish factual accuracy from fluency, confidence, or style.
• High attention to detail and comfort following structured guidelines.
• Reliable, self-directed, and responsive in an asynchronous remote environment.

Preferred Qualifications

• Experience in search quality, fact-checking, content evaluation, trust and safety, annotation, QA, or prompt/rubric writing.
• Familiarity with search evaluation concepts such as factuality, helpfulness, severity, side-by-side comparisons, or tool-use assessment.
• Experience working with LLM evaluation workflows or human data projects.
• Multilingual skills are a plus.
• Bachelor’s degree preferred; advanced degree or strong professional background is a plus.