All jobs
OpsLabel logo

AI Evaluation Specialist - RLHF, SFT & MCP (Remote, Immediate Joiners)

OpsLabel · Posted 1 month ago

  • India (Remote)
  • Remote
  • Contract

About the role

AI Evaluation Specialist - RLHF, SFT & MCP (Remote, Immediate Joiners)

OpsLabel is expanding its expert network, and we have an immediate need for AI evaluation specialists. We need your judgment on frontier AI projects spanning RLHF, SFT, and MCP (agent tool-use) evaluation, helping train and stress-test how well models respond, reason, and use tools.

  • Remote (India preferred)
  • Freelance / Contract, long-term project work
  • Open to strong technical minds: ML, engineering, linguistics, or a STEM domain.
  • Advanced degree a plus, not required. What matters is sound judgment on what a good AI answer looks like.

Must have

  • Strong judgment on the quality, correctness, and safety of AI model outputs
  • Clear written reasoning in English
  • Comfort working to detailed rubrics and guidelines
  • A technical or analytical background (ML, software, data, linguistics, or a STEM field)

Nice to have

  • Hands-on experience with RLHF, preference ranking, or reward modeling
  • SFT data creation: instruction-response pairs or chain-of-thought
  • Familiarity with MCP, function or tool calling, or agent workflows
  • Programming or prompt-engineering background

What you'll do

  • Rank and compare model responses to build high-quality preference data
  • Write and review instruction-response and chain-of-thought examples
  • Evaluate agent tool-use: tool-calling accuracy and multi-step workflows
  • Catch the subtle errors and reasoning failures generic reviewers miss
  • Deliver high-quality work across multiple sprints with direct feedback from senior reviewers

Why this role matters

  • Your judgment directly shapes how the next generation of AI responds, reasons, and uses tools. This is high-impact, flexible work for experts who want their depth to count, not faceless annotation work.Engagement model:Flexible hours, work on your own schedule
  • Long-term engagements on frontier AI projects
  • Multiple delivery sprints across 3 to 6+ months

How to apply: Submit your CV along with a short note covering your technical background, any RLHF, SFT, or MCP/agent experience, and your programming or prompt-engineering familiarity. You'll be added to our expert pool and matched to the projects that best fit your background.

Apply here: https://opslabel.in/r/ai-evaluation-specialist-rlhf-sft-mcp

#RLHF #SFT #AIEvaluation #FreelanceJobs #RemoteWork

Skills

AI evaluationModel ranking

Quark9 guide

How to land this AI Evaluation Specialist - RLHF, SFT & MCP (Remote, Immediate Joiners) role

Our notes for applicants, based on how data and AI hiring usually works and on 5 similar open jobs on Quark9.

Resume checklist for this job

Most companies screen resumes with an ATS before a person reads them. Make sure each skill you genuinely have appears in your resume in the same words the job uses.

Must have

  • AI evaluation
  • Model ranking
  • Lead each project with the business result (revenue, cost, time saved), then the method
  • List the stack precisely: SQL dialect, Python libraries, BI tool, cloud platform
  • Link a portfolio or GitHub with one or two well-documented projects
Check your resume against this job free

How data and AI interviews usually run

  1. A screening call on your projects and the size of data you have worked with
  2. A SQL round, usually joins, window functions and aggregations on a sample schema
  3. A case or modelling round: frame a business problem, choose metrics, explain trade-offs
  4. A take-home or live exercise in Python, then a final round with the hiring manager

Questions worth preparing

  • Write a query for the second-highest salary in each department.
  • How would you measure whether a new feature improved retention?
  • Explain overfitting to a non-technical stakeholder and how you guard against it.
  • Walk through a project end to end: the question, the data, the method and the impact.

Every company runs its own process, so treat this as a likely shape, not a promise. Practise a mock interview on Quark9 and get scored feedback before the real one.

The market for AI Evaluation Specialist roles right now

  • Most openings are in Mumbai, Pune, Delhi, and 4 of the 5 we looked at offer remote work.

From open jobs on Quark9 matching “AI Evaluation Specialist”, updated every few minutes.

Similar jobs