Compare the confidence intervals.
A small lead may not separate two models. Read the uncertainty alongside the score, and add more representative examples when the evidence is thin.
01 / Evaluation
Compare models on your team’s examples.
Run candidates against the same examples and scoring criteria. Inspect the answers behind the scores before choosing a model for your task.
Your data sets the standard.
Try a model comparison
Choose a task
The Orlo workspace
Define what a good answer must include.
For support routing, every answer needs a category, priority, and escalation decision. Keep those rules beside the examples your team has reviewed.
Support ticket routing
Route inbound support tickets to a category and priority, and decide whether to escalate to a person.
Live: release v1 on claude-sonnet-4-20250514
Live
Review
Standard
Sources
Changes
Task version 1, with 6 examples in support-golden-set-v1 and 1 rule. Comparisons test models against it.
Rules
Every answer has category, priority, and escalate.
Required fields
Error
When an answer breaks a rule, Orlo still returns it and marks it with a warning.
Examples
6 examples in the latest version: account_access, api_outage, billing_upgrade, model_behavior, security_incident, and documentation.
support-golden-set-v1
Dataset version 1
6 examples
Small score differences may need more evidence.
Use corrections in the next comparison.
Have your reviewers inspect disagreements. Their corrected answers can become examples in the next comparison.