Dataset
Mathematical problem solving remains a challenging test of reasoning for large language and multimodal models, yet existing benchmarks are limited in size, language coverage, and task diversity. We introduce MathNet, a high-quality, large-scale, multimodal, and multilingual dataset of Olympiad-level math problems together with a benchmark for evaluating mathematical reasoning in generative models and mathematical retrieval in embedding-based systems.
MathNet spans 47 countries, 17 languages, and two decades of competitions, comprising 30,676 expert-authored problems with solutions across diverse domains. In addition to the core dataset, we construct a retrieval benchmark consisting of mathematically equivalent and structurally similar problem pairs curated by human experts.
MathNet supports three tasks: (i) Problem Solving, (ii) Math-Aware Retrieval, and (iii) Retrieval-Augmented Problem Solving. Experimental results show that even state-of-the-art reasoning models (78.4% for Gemini-3.1-Pro and 69.3% for GPT-5) remain challenged, while embedding models struggle to retrieve equivalent problems. We further show that RAG performance is highly sensitive to retrieval quality; for example, DeepSeek-V3.2-Speciale achieves gains of up to 12%, obtaining the highest scores on the benchmark. MathNet provides the largest high-quality Olympiad dataset together with the first benchmark for evaluating mathematical problem retrieval, and we publicly release both the dataset and benchmark.
Here are some examples from MathNet. The dataset contains standard text-only problems, diagram-heavy geometry, and retrieval pairs where the challenge is not exact lexical matching but mathematical equivalence or structural resonance.
MathNet-RAG pairs problems by three levels of mathematical similarity — from strict equivalence to loose thematic clustering.
| Mode | Problem A | Problem B |
|---|---|---|
| Invariance | ||
| Syntactic Equivalence | Find $f:\mathbb{R}\to\mathbb{R}$ such that $f(x^2-y^2)=(x-y)(f(x)+f(y))$. | Find $g:\mathbb{R}\to\mathbb{R}$ such that $(g(a)+g(b))(a-b)=g(a^2-b^2)$. |
| Reformulation | Let $a_i>0$. Prove $\sum_{i=1}^n \frac{a_i}{a_i^2+a_{i+1}a_{i+2}} \le \sum_{i=1}^n \frac{1}{a_i+a_{i+1}}$. | Let $a_i>0$. Prove $\sum_{i=1}^n \frac{a_i^2}{a_i^2+a_{i+1}a_{i+2}} \ge \tfrac{1}{2}$. |
| Transformational | Find all $x\in\mathbb{R}$ such that $4^x+6^x=9^x$. | Find all $x\in\mathbb{R}$ such that $(2/3)^x+(3/2)^x=5/2$. |
| Structural Resonance | ||
| Generalization | For $k\ge1$, prove that $k$ divides $\binom{n}{k}$ for all $n\ge k$. | Show $\binom{n}{m}\equiv\prod\binom{n_i}{m_i}\pmod{p}$, where $n=\sum n_i p^i$, $m=\sum m_i p^i$. |
| Common Lemma | If $ab+1\mid a^2+b^2$, show $\frac{a^2+b^2}{ab+1}$ is a perfect square. | If $a^2+b^2+c^2=k(ab+bc+ca)$, show $k\in\{1,2,3\}$. |
| Structural Reduction | Prove that $4^n+2^n+1$ is never prime. | Prove $2^{2n}+2^n+1$ is divisible by $3$ for all $n$. |
| Affinity (Thematic) | ||
| Affinity | Show the largest prime factor of $\binom{2n}{n}$ is greater than $n^{2/3}$. | For every $n>1$, there exists a prime $p$ with $n<p<2n$. |
Table: Three levels of mathematical similarity, illustrated with Olympiad examples. Invariance means the problems are equivalent under reformulation. Structural Resonance means they share key lemmas or reductions. Affinity is a looser connection — same topic area, different approach.
MathNet supports three tasks. The first tests whether models can solve Olympiad problems outright. The second tests whether embedding models can retrieve mathematically equivalent problems from a large pool. The third combines both: does giving a model a similar problem as context actually help?
MathNet covers 47 countries and 17 languages, with problems spanning two decades of competition math. The solutions are long (considerably longer than those in existing benchmarks), which is part of what makes evaluation harder.
MathNet dataset statistics. (a) Contest type distribution. (b) Solution length vs. existing benchmarks — MathNet solutions are much longer. (c) Problems per year. (d) Topic and sub-topic distribution. (e) Language distribution: 74% English, 26% non-English across 17 languages.
Each problem starts as a scanned competition booklet. We run OCR, split the text into problem–solution pairs, normalize the formatting, and have human experts verify the output before anything enters the dataset.
Data extraction and curation pipeline. Competition PDFs are converted to markdown via OCR, split into problem–solution blocks, normalized with GPT-4.1, and verified by human experts.
The top solving model hits 78.4%, which's very strong; howerver, retrieval is the bigger gap: Recall@1 stays below 5% for every model we tested. Expert-retrieved context helps solving accuracy, but only when the retrieval is actually good.
MathNet-Solve-Test
Problem Solving on MathNet-Solve-Test (6,400 problems). The chart uses the paper's overall micro-average accuracy values to make the ranking legible on the page.
Takeaway: LMMs with reasoning are clearly strongest overall, but even the top model remains well below perfect performance.
MathNet-Retrieve
Math-Aware Retrieval on MathNet-Retrieve (10,000 anchor problems). This chart uses the paper's aggregate “All” Recall@1 and Recall@5 values.
Takeaway: Recall@1 stays very low even for the best models, while Recall@5 is much stronger, showing that mathematically equivalent retrieval is still unreliable at shallow depths.
MathNet-RAG
Retrieval-Augmented Problem Solving on MathNet-RAG (35 problems). These bars highlight the most legible comparisons from the paper across zero-shot, Embed-RAG, and Expert-RAG settings.
Takeaway: expert retrieval most often gives the strongest gains, but improvements remain model-dependent and grading-dependent.
If you use MathNet in your work, please cite the paper.
@inproceedings{alshammari2026mathnet,
title = {MathNet: A Global Multimodal Benchmark for Mathematical
Reasoning and Retrieval},
author = {Alshammari, Shaden and Wen, Kevin and Zainal, Abrar and
Hamilton, Mark and Safaei, Navid and Albarakati, Sultan and
Freeman, William T. and Torralba, Antonio},
booktitle = {International Conference on Learning Representations},
year = {2026},
url = {https://mathnet.mit.edu}
}
For questions about the dataset, benchmark, or paper, reach out to shaden@mit.edu.