Red-teaming an LLM isn't about trying to make it say something offensive (though that's part of it). It's about systematically probing the model's failure modes — the places where it produces output that looks right but isn't, or where it confidently answers questions it shouldn't.
The failure modes annotators flag
1. Plausible but wrong
The model generates a response that reads fluently, uses appropriate technical vocabulary, and cites something that sounds like a real source — but the information is fabricated. This is the hardest category to evaluate because it requires domain knowledge to catch.
2. Instruction non-compliance
The user asks for a bulleted list, and the model responds with paragraphs. Or the user asks for the answer in under 50 words, and the model writes 200. These failures are easy to flag but important for downstream evaluation: a model that ignores formatting instructions will frustrate users.
3. Unsafe output under edge-case prompts
Standard harmful prompts are well-guarded against. Red-teaming focuses on edge cases: multi-turn manipulation, role-play scenarios that gradually shift boundaries, or ambiguous prompts where the "safe" response is unclear.
4. Inconsistency across turns
The model says one thing in turn 3 and contradicts itself in turn 7. In long conversations, consistency degrades. Annotators track whether the model maintains its positions, facts, and persona across a multi-turn session.
What makes a good red-teaming annotator
- Domain expertise: A medical red-teamer needs to know when a dosage recommendation is plausible but wrong. A legal red-teamer needs to catch when the model cites a statute that doesn't exist.
- Adversarial creativity: Good red-teamers think in attack vectors. They don't just test the obvious — they test the subtle.
- Rubric discipline: Every flag needs to map to a specific failure category in your rubric. "This feels off" isn't actionable. "Plausible hallucination in medical context, severity: high" is.
How we structure red-teaming engagements
- Define failure taxonomy with your team (safety, accuracy, compliance, consistency)
- Match annotators by domain expertise and adversarial testing experience
- Run structured probing sessions with prompt templates and free-form exploration
- Deliver failure reports with severity scoring, reproducible prompts, and model response captures
Need to red-team your model before launch? Start a project — tell us your model, your risk profile, and your domain, and we'll build a red-teaming protocol.