EntropyMath Leaderboard
A high-entropy mathematical reasoning benchmark for LLMs
3 runs per problem · Errors included · Problem & response details available
- Accuracy counts correct answers across all three attempts. Pass@3 counts problems with at least one correct answer. Errors and timeouts count as incorrect.
- Token averages use only runs with reported usage; missing usage is not treated as zero.
- Select a result cell for the recorded problem, reference answer, model response, and all three attempts. Missing responses and usage remain explicitly unreported.
- This is the mathematics subject, in English—not the CSAT Korean language/verbal reasoning evaluation. Motif 3 was evaluated manually in web chat.
- Kimi K3: after an initial retry of the four original timeout slots, 1 completed response was retained and the remaining 3 slots were retried via DeepInfra with provider fallbacks disabled. The same prompt, sampling settings, and 8192-token limit were used. Completed responses replace only the original error slots; any failed retries remain errors. These are not extra best-of runs. Original error and retry records are preserved separately.
- Kimi K3: 3 latest retry requests reached the 8192-token limit without a final response and remain errors. No token-budget increase has been applied.
Model Accuracy vs Pass@3
100%
75%
50%
25%
0%
Kimi K3
Solar Pro 4
K-EXAONE-2
Motif 3
Accuracy
Pass@3
Avg Tokens per Reported Run
2.4K
1.8K
1.2K
602.915579710145
0
Solar Pro 4
K-EXAONE-2
Kimi K3
Avg Tokens / Problem
Entrance Exams brings university entrance examinations together, including verbal reasoning and mathematics. Results remain separate by exam, year, subject, and evaluation language.
Results are reported using Pass@3 metrics to account for generation variance. Detailed execution traces are available for transparency.
Performance Legend
Mastery (100%)
3/3
Strong (66%)
2/3
Weak (33%)
1/3
Fail (0%)
0/3
Leaderboard / CSAT 2026 · Mathematics · English
Change benchmark ↑Scroll horizontally to see all problems. Select a result cell to view the problem and recorded response.
TL = token limit reached before a final answer. These runs still count as incorrect.
| Model | Acc | Pass@3 | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| API / Others | ||||||||||||||||||||||||||||||||||||||||||||||||
Kimi K3API · no tools3 token limits · Tokens: 135/138 runs | 92.0 | 95.7 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 2/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 2/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 0/3 | 3/3 | 0/3TL ×1 | 2/3 | 3/3 | 3/3 | 3/3 | 3/3 | 1/3TL ×2 | 3/3 | 3/3 | 3/3 |
| K-LLM Project Round 3 | ||||||||||||||||||||||||||||||||||||||||||||||||
Solar Pro 4API · no tools0 errors · Tokens: 138/138 runs | 77.5 | 84.8 | 2/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 0/3 | 3/3 | 3/3 | 3/3 | 2/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 0/3 | 0/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 2/3 | 3/3 | 0/3 | 3/3 | 3/3 | 3/3 | 1/3 | 1/3 | 3/3 | 0/3 | 0/3 | 3/3 | 3/3 | 3/3 | 3/3 | 0/3 | 3/3 | 1/3 | 2/3 |
K-EXAONE-2API · no tools0 errors · Tokens: 138/138 runs | 65.9 | 78.3 | 1/3 | 3/3 | 3/3 | 3/3 | 3/3 | 1/3 | 3/3 | 3/3 | 3/3 | 1/3 | 3/3 | 3/3 | 3/3 | 1/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 0/3 | 0/3 | 0/3 | 3/3 | 3/3 | 3/3 | 3/3 | 3/3 | 2/3 | 2/3 | 0/3 | 2/3 | 2/3 | 3/3 | 2/3 | 0/3 | 2/3 | 0/3 | 0/3 | 3/3 | 3/3 | 2/3 | 3/3 | 0/3 | 1/3 | 0/3 | 0/3 |
Motif 3Manual web chat · medium response level0 errors · Tokens not reported | 49.3 | 73.9 | 0/3 | 2/3 | 2/3 | 1/3 | 2/3 | 0/3 | 3/3 | 3/3 | 3/3 | 1/3 | 3/3 | 3/3 | 3/3 | 1/3 | 0/3 | 2/3 | 3/3 | 2/3 | 2/3 | 1/3 | 1/3 | 0/3 | 2/3 | 0/3 | 2/3 | 3/3 | 1/3 | 1/3 | 1/3 | 1/3 | 3/3 | 3/3 | 1/3 | 3/3 | 0/3 | 0/3 | 0/3 | 0/3 | 2/3 | 3/3 | 0/3 | 0/3 | 0/3 | 1/3 | 1/3 | 2/3 |