Background
The story
The AIMO Proof Pilot was the first competition focused solely on fully open-source LLMs that solve Olympiad-level math problems. The models that came out of that competition showed strong gains, rivalling Opus 4.8. After the competition concluded, Frieder from the AIMO and the Fields Model Initiative, Sam from AIMO, the top three winning teams, and Michal and Adam from NII all joined forces to provide further compute and do further work on the winning model. The improved model was uploaded before IMO 2026 took place and then evaluated on IMO problems in the cleanest evaluation to date on IMO problems. The harness and the full code is public.
This effort is the first time that a fully open-source LLM achieves a medal at the International Mathematical Olympiad 2026 (IMO 2026). This has for a long time been a cornerstone for mathematical evaluation, and the series of AI Math Olympiads (AIMO) have as their objective to support the creation of these models. The current result in fact builds upon the latest AIMO competition, the AIMO Proof Pilot.
Significance
Why this is a big deal
For the first time it is possible to do detailed post-mortems. We hope that this will inspire other people to build more on our work and that this kickstarts new data attribution efforts and introspection efforts.
While the first landmark result was the AIMO2 × OpenAI evaluation in March 2025, a flood of further news followed with various claims about LLMs being able to solve IMO-level problems. Most of these are very hard to verify. In particular, a lot of nuance is not shared: How expensive is it to carry these evaluations out? What hardware does one need?
From the point of view of science and scientific reproducibility, most of these were less rigorous than the AIMO2 × OpenAI evaluation, which had more unseen, difficult Olympiad-level problems than the IMO (50 instead of 6), and a controlled time span of several hours. Due to the large number of evaluated problems, it would have been very hard to, e.g., adjust some settings on the fly during the evaluation to improve performance. The AIMO2 × OpenAI evaluation was therefore the first point in time that members outside of AI companies could see that LLMs were able to solve unseen math problems.
Evaluation Transparency Audit · IMO 2026
Reproducibility Comparison
| # | Model / System | Score | Medal | Open weights before eval? | Exact checkpoint public before eval? | Proofs shared? | Reproducibility |
|---|
| 1 | Huawei Celia | 42/42 | Gold | No | N/A | No | E |
| 2 | RedNote / Xiaohongshu dots-note-3.0 | 42/42 | Gold | No | N/A | Partial (PDFs) | E |
| 3 | NVIDIA Nemotron 3 Ultra ensemble | 30/42 | Gold | Partial | No | No | E |
| 4 | Meta internal Muse Spark | Gold (top 4%) | Gold | No | N/A | No | E |
| 5 | Claude Opus 5 — Anthropic eval | 42/42 | Gold | No | N/A | Partial | C |
| 6 | Claude Fable 5 — Deedy eval | 42/42 | Gold | No | N/A | Yes | C |
| 7 | GPT-5.6 Sol xhigh — Deedy eval | 39/42 (42/42 after repair) | Gold | No | N/A | Yes | C |
| 8 | GPT-5.6 Sol Pro xhigh — Deedy eval | 37/42 | Gold | No | N/A | Yes | C |
| 9 | GPT-5.6 Sol max — Deedy eval | 30/42 | Gold | No | N/A | Yes | C |
| 10 | GPT-5.6 Sol default — Deedy eval | 28/42 | Silver | No | N/A | Yes | C |
| 11 | Kimi K3 — Deedy eval | 36/42 (42/42 after repair) | Gold | No | N/A | Yes | C |
| 12 | Meta Muse Spark 1.1 — Deedy eval | 26/42 | Silver | No | N/A | Yes | D |
| 13 | DeepSeek V4 Pro — Deedy eval | 19/42 | Bronze | Yes | No | Yes | C/D |
| 14 | Claude Sonnet 5 + AutoFyn | 35/42 | Gold | No | N/A | No | C |
| 15 | GLM-5.2 + AutoFyn | 34 / 34 / 28 (three runs) | Gold / Silver | Yes | No | Partial | C |
| 16 | FM-Pochi-32B (ours) | Bronze | Bronze | Yes — fully open | Yes | Yes | A |
Reproducibility grades: A = fully self-contained and reproducible; C = protocol documented, manual reconstruction required; D = outputs auditable, generation not plug-and-play; E = effectively non-reproducible.