PhysicsWallah’s Aryabhata 1.0 Scores 90.2% in JEE — Outperforms Open-Weight AI Models, Trails Frontier Leaders
 |
| PhysicsWallah |
PhysiscsWallah, an ed-tech startup has developed and launched a small language model ((SLM) named as "Aryabhata 1.0",a 7-billion parameter LLM containing 4000 tokens. The model is named after ancient Indian mathematician. The naming of the model reflects India's longstanding heritage in mathematical advancements.
 |
| Aryabhata 1.0 Math Benchmarks |
How the model is built?
Physicswallah AI Research division started by merging several models including Qwen 2.5 Math, an Alibaba backed open model, Ace Math, a Nvidia tuned Qwen model for mathematical benchmarking, DeepSeek R1 Distill for reasoning, which is then trained on 130k questions distilled from PhysicsWallah's database. The company stated that it used in-house Group Relative Policy Optimiation (GRPO) as a math specific reward function, removing KL divergence penalty (An approach designed to limit how much the learned policy or model diverges from a baseline policy or distribution) and clipping.
How was the model tested?
The model is tested with official JEE Mains 2025 mathematics papers. it had 19 papers totaling 475 questions. The model needed to answer
- multiple choice questions with one correct option
- Numeric Answer Type (NAT) questions requieing precise numerical response
Accuracy and Comparison against other models
 |
| Aryabhata 1.0 model accuracy comparison among question paper set |
Aryabhata 1.0 is able to deliver 90% accuracy beating other open models currently available, and stands on par with frontier models in the market, despite using very conservative model token size of 4k.
 |
| Model Accuracy Vs Token Usage |
Proposed Use cases for the model
The company has released its models available for competitive exam preparation, doubt-solving, educational tutoring and concept explanation.
Future roadmap for model development
The company is planning to develop the model by extending it to include physics, chemistry domain, supporting the study for competitive exams like JEE Advanced, NEET. It also aims at improving the efficiency, affordability and accuracy for real-time deployments.
AI development scenario in India
Companies like Sarvam.AI is developing large model with the support of compute power loan from government of India. Several companies including Zoho and others are devleoping smaller models that can fit niche usage application for Indian context. Though Indian companies may take years to reach the current level of the US and China, they are clearly beginning to move in that direction.
Comments
Post a Comment