Human evaluation and RLHF data for language models

Rubric scoring, preference pairs, SFT demonstrations and red teaming by trained human reviewers. Every judgment comes with a reason you can read.

Start with a pilot batch See task types

What we deliver

Use one task type or chain several, for example SFT demonstrations first, then preference pairs on the fine-tuned model.

Rubric-based evaluation

Reviewers score each response against criteria you define: accuracy, completeness, instruction following, tone, safety.

Pairwise preference

Two responses, one prompt. Reviewers pick the better one and write a short reason. The core input for reward models.

Ranking

Three or more responses ordered best to worst, for reward modeling and model comparisons.

SFT demonstrations

Prompts and ideal responses written from scratch to your style guide, for supervised fine-tuning.

DPO pairs

Prompt, chosen response and rejected response, with the reason, in the exact format your trainer expects.

Red teaming

Adversarial prompts that try to break your policies: jailbreaks, prompt injection, unsafe advice, data leaks.

Safety and policy labeling

Responses tagged against your content policy, with severity levels and the policy clause that applies.

RAG and citation checks

Is the answer supported by the retrieved sources? Reviewers flag unsupported claims and wrong citations.

Agent and tool-use evaluation

Did the agent pick the right tool, pass the right arguments and reach the goal? Step-by-step trace review.

Regression testing

The same prompt set scored on each new model version, so you see what got better and what broke.

An example rubric

Subjective work needs written definitions. This is the kind of rubric we agree with you before any scoring starts. Yours will match your product and policies.

CriterionWhat reviewers checkScale
AccuracyFacts, numbers and steps are correct. Unverifiable claims are flagged, not guessed.1 to 5
Instruction followingEvery part of the prompt is answered, in the requested format and length.1 to 5
HelpfulnessThe user can act on the answer without asking again.1 to 5
Safety and policyNo harmful content. Refuses only when the policy says so, and explains briefly.Pass / fail + clause
ToneMatches your brand voice. No filler, no over-apologising.1 to 5

How a project runs

The same pilot-first process as all NORBEON work, with one extra step for subjective tasks: calibration.

  1. Share the task and examples

    Your prompts or prompt themes, model outputs, policies and what a good answer looks like.

  2. Write the rubric together

    Criteria, scales and edge cases with worked examples. You sign it off.

  3. Calibration round

    All reviewers score the same set. We compare, discuss disagreements and tighten the rubric.

  4. Pilot batch and QA report

    A small batch with agreement scores, error notes and rationales you can audit.

  5. Scale

    Full delivery in batches, with gold items mixed in to catch drift.

Keeping human judgment consistent

Two people can read the same answer differently. These checks are built into every project.

  • A reason for every judgmentNo bare scores. You can read why each response won or lost.
  • Agreement between reviewersA share of items is scored twice, and we report how often reviewers agree.
  • Hidden gold itemsPre-scored items are mixed into every batch to catch drift and fatigue.
  • Order and length bias checksResponse order is randomised, model names are hidden, and we watch for reviewers who simply prefer longer answers.
  • Escalation, not guessingUnclear cases go to a lead reviewer and, if needed, to you. The decision is added to the rubric.

Formats and tools

We work in your platform or set one up, and deliver in the shape your training code reads.

Ways of working

Batch transcript ratingLive chat with your modelSingle-turnMulti-turn conversations

Delivery formats

JSONLDPO: prompt, chosen, rejectedRanked listsRubric scores + rationaleCSV

Tools

Label StudioArgillaLabelboxYour in-house tool

Rules our reviewers follow

Human feedback is only useful if it is actually human and actually private.

  • No AI tools. Reviewers may not use chatbots to write, rewrite or judge responses.
  • NDA first. Signed before prompts, outputs or policies are shared.
  • Need-to-know access. Only reviewers on your project can see your data.
  • Model names hidden. Reviewers judge outputs, not brands.
  • Used only for your project. Prompts and outputs are never reused or shared.

Where teams use it

Most projects start with one of these.

Chat and support assistants

Helpfulness, tone and policy checks on real customer questions.

Search and RAG

Answer quality, groundedness and citation accuracy against retrieved documents.

Coding assistants

Correctness, explanation quality and preference pairs on code answers.

Agents and workflows

Tool choice, argument accuracy and task completion across multi-step traces.

Model launches

Side-by-side comparison of your new model against the current one before release.

Safety reviews

Red teaming and policy labeling before a public launch or a policy change.

Questions teams ask

If yours isn't here, ask in the form below.

How is LLM evaluation priced?

Per task or per hour, depending on prompt length, the number of responses per prompt and how much expertise the task needs. We quote after seeing samples and confirm the rate after the pilot.

Do you write the prompts too?

Yes. We can write prompt sets to your themes and difficulty mix, or work only on prompts and outputs you provide.

Can tasks need subject knowledge, like code or finance?

Yes. Tell us the domain in the form. We match the task to reviewers with relevant background and confirm this with you before the pilot.

Which languages do you support?

Tell us the languages and locales you need. We confirm reviewer availability for each one before the pilot, rather than promising it upfront.

How do you stop reviewers from favouring longer answers?

The rubric defines when length helps and when it doesn't, response order is randomised, and we check each reviewer's choices for length and position patterns.

Do you also label images, text or audio?

Yes. See image and video, text and NLP, audio and speech and data collection.

Start with a pilot batch

Tell us about your model and what you want to measure or improve. We'll reply with questions, a draft rubric outline and a pilot plan.

Prefer email? Write to hello@norbeon.com.

Are you a reviewer or annotation team? Register as a vendor.

By sending this form, you agree to our Privacy Policy.