LLM Evals

Patronus AI vs OpenEvals

Patronus AIOpenEvals

47%

53%

Leading: OpenEvals (53.3%)

Insufficient data

This matchup has 15 decisive cases (minimum 30 required for publication).

Metric	Value
Patronus AI wins	7
OpenEvals wins	8
Abstains (no tool)	237
Other tool chosen	4276
Decisive cases	15
Patronus AI win rate (unweighted)	46.7%
95% CI	24.8% - 69.9%
Patronus AI win rate (weighted)	46.7%

Verified critics can leave comments here.

Verified critics can leave comments here.

Model range	Tier	Patronus AI	OpenEvals	None	Other	A rate
DeepSeek Flash 3.2–4.1	Mid	0	0	26	210	n/a
DeepSeek Pro 4–4 0813	Frontier	0	0	2	106	n/a
DeepSeek R1	Frontier	0	0	12	228	n/a
Devstral	Mid	0	0	19	182	n/a
Gemini Flash 2.5–3.8	Small	0	0	12	223	n/a
Gemini Pro 2.5–3.1 Preview	Frontier	2	0	17	219	100%
GLM 5 Turbo–5.3	Frontier	0	0	19	221	n/a
GPT 5.4–6 Astra	Frontier	1	0	5	234	100%
GPT Coding 5.3 Codex–6.1 Sol	Frontier	0	0	2	238	n/a
GPT Mini 5.4 Mini–6 Luna	Mid	0	0	9	228	n/a
Haiku	Small	0	0	4	227	n/a
Kimi 2.5–3	Frontier	0	0	5	221	n/a
Llama Maverick	Frontier	0	8	18	205	0%
Llama Scout	Small	0	0	29	185	n/a
MiMo Pro 2–2.6	Frontier	4	0	17	218	100%
MiniMax M 2.7–3	Frontier	0	0	12	224	n/a
Mistral Small	Mid	0	0	5	221	n/a
Opus 4.6–5.5	Frontier	0	0	9	230	n/a
Qwen Coder	Mid	0	0	4	227	n/a
Sonnet 4.6–5.5	Frontier	0	0	11	229	n/a

Prompt	Tier	Patronus AI	OpenEvals	None	Other	A rate
ai-revenue-ops-copilot	Advanced	4	0	3	567	100%
ai-engineering-workflow	Intermediate	0	4	1	168	0%
ai-agent-application	Intermediate	0	3	1	171	0%
ai-support-agent-platform	Advanced	2	0	5	580	100%
ai-support-agent-platform	Beginner	1	0	80	503	100%
ai-revenue-ops-copilot	Beginner	0	1	13	570	0%
ai-agent-application	Advanced	0	0	0	173	n/a
ai-agent-application	Beginner	0	0	40	138	n/a
ai-engineering-workflow	Advanced	0	0	1	167	n/a
ai-engineering-workflow	Beginner	0	0	82	90	n/a
ai-revenue-ops-copilot	Intermediate	0	0	5	575	n/a
ai-support-agent-platform	Intermediate	0	0	6	574	n/a