Practitioner brief · model risk & AI governance
Five frontier LLMs read 789 hand-labeled filings paragraphs. The best scored 69%.
A plain-language brief on what a ground-truth benchmark of ChatGPT-4o, Claude, Gemini, Grok, and DeepSeek reveals about LLM accuracy in ESG disclosure classification — and what it means for your model inventory.
What the brief covers — in findings-table form
Domain model vs. frontier LLM accuracy83% vs 59–69%Finding 1
Model consensus vs. human ground truthagreement ≠ correctnessFinding 2
Error concentration in minority classesF1 0.87 vs 0.37Finding 3
Validation battery for LLM classifiers6 componentsIncluded