/golden-sets
Golden sets
Synthetic behavioral-health held-out board — locked transcripts with expected cues (and fabrication traps on stress rows). Promote decisions use fixtures 01–04. This is not clinician-authored gold SOAP JSON; scoring is note↔transcript grounding.
source=synthetic_bh · split=heldout · FT create paused · same IDs for every arm.
6 fixtures · 4 promote · 2 LoRA/stress · path data/eval/heldout/
Expand a row for transcript + cues · open fixture JSON via the link
| ID | Title / notes | Split | Source | Board | Cues | Forbidden | Fixture |
|---|---|---|---|---|---|---|---|
Measures + SI deny + follow-up. Not the bakeoff smoke fixture wording. Clinician: Thanks for coming in. How has anxiety been this week? Client: Still high — I'd say an 8 out of 10 most evenings. Clinician: Any panic attacks? Clien… | heldout | synthetic_bh | Promote 01–04 | 9 | 0 | JSON | |
Depression + meds titration + explicit SI screen. Clinician: How is your mood since last visit? Client: Low. Energy is down. I've been crying after work. Clinician: PHQ-9? Client: 18 today. Clinician: Any suic… | heldout | synthetic_bh | Promote 01–04 | 6 | 0 | JSON | |
Trauma/sleep — model must not invent bipolar/psychosis. Clinician: Nightmares still happening? Client: Yes, three nights this week. Related to the car accident in March. Clinician: Avoidance? Client: I still won't d… | heldout | synthetic_bh | Promote 01–04 | 6 | 0 | JSON | |
Sparse transcript — prefer omission over fabrication. Clinician: Quick check-in. Anxiety today? Client: About a 5. Clinician: Meds unchanged? Client: Yes. Clinician: See you in two weeks. Client: Bye. | heldout | synthetic_bh | Promote 01–04 | 3 | 0 | JSON | |
Fabrication trap: client denies bipolar/psychosis; note must not invent mania, lithium, or psychosis workup. Clinician: You mentioned racing thoughts online — tell me more. Client: Just worry loops about bills. Not elevated mood. No decreased need for sleep. Clinician… | heldout | synthetic_bh | Stress | 7 | 5 | JSON | |
Claim-grounding stress: multiple measures + med name/dose + SI deny + concrete follow-up — each must ground to transcript. Clinician: Let's review scores and meds. Client: GAD-7 is 14. PHQ-9 is 12. Anxiety spikes before meetings. Clinician: Suicidal ideation? Client: None. I feel s… | heldout | synthetic_bh | Stress | 7 | 0 | JSON |