Introduction
Scaling laws have become the gravitational constants of modern AI development, shaping how billions of dollars in compute budgets get allocated across the world's most ambitious training runs. Two frameworks dominate the conversation: the original Kaplan et al. scaling laws published by OpenAI in 2020, and the revised Chinchilla scaling laws from DeepMind in 2022. Both attempt to describe how loss decreases as you scale compute, parameters, and data, but they arrive at fundamentally different conclusions about where to invest marginal FLOPs. For engineers and researchers planning production training runs, choosing the wrong framework can mean burning millions in compute on a suboptimal configuration. The disagreement between these two models is not academic trivia; it has reshaped how every major AI lab in the United States and beyond approaches the tradeoff between model size and training data volume.
The Core Disagreement: Parameters vs. Data
Understanding the divergence between the Kaplan and Chinchilla frameworks requires looking at what each predicts about the relationship between compute and performance. Both agree that LLM scaling laws follow power-law relationships, but they disagree sharply on the optimal allocation of a fixed compute budget between model parameters and training tokens.
What Kaplan et al. Actually Claimed
The original 2020 paper from OpenAI established several influential claims about neural network scaling behaviour. The core argument was that model performance (measured as cross-entropy loss) scales as a power law with respect to compute, dataset size, and parameter count, and that the parameter axis dominates. Under Kaplan's framework, given a fixed compute budget, you should train a very large model on a relatively modest amount of data.
Parameter dominance: Kaplan found that increasing model size yielded steeper loss improvements per unit of compute than increasing training data proportionally.
Recommended ratio: The framework implied roughly a 1:1 ratio between parameters and training tokens, meaning a 10B parameter model would be trained on approximately 10B tokens.
Early stopping preference: Because bigger models learn faster per step, Kaplan advocated training large models for fewer steps rather than training smaller models to convergence.
Smooth power laws: Loss curves were presented as remarkably smooth and predictable across seven orders of magnitude, suggesting these relationships could be extrapolated reliably.
Where Chinchilla Changed the Equation
DeepMind's 2022 Chinchilla paper reran the analysis with a more rigorous experimental design and reached a strikingly different conclusion. Instead of parameter counts being the primary lever, the Chinchilla scaling laws showed that computing optimal training requires roughly equal scaling of both parameters and training tokens. Specifically, the optimal number of training tokens should scale linearly with model size, at approximately 20 tokens per parameter. A 70B parameter model, under Chinchilla's framework, should see roughly 1.4 trillion tokens during training. This single insight invalidated the common practice of training massive, undertrained models and redirected the industry toward data scaling as a co-equal priority.
Empirical Evidence and Real-World Validation
Theoretical frameworks only matter if they survive contact with real training runs. Since 2022, the AI scaling research community has accumulated enough empirical evidence to assess which set of predictions holds more reliably across diverse training configurations and model families.
Industry Training Runs as Ground Truth
The strongest evidence in favour of the Chinchilla framework came from the model itself. Chinchilla (70B parameters, 1.4T tokens) outperformed Gopher (280B parameters, 300B tokens) on virtually every benchmark despite using the same compute budget. This was a direct, controlled demonstration that Kaplan's parameter-heavy allocation was leaving performance on the table. Meta's LLaMA family reinforced this: LLaMA-1 (65B) was trained on 1.4T tokens and matched or exceeded much larger models trained under Kaplan-style assumptions.
The trend continued with LLaMA-2 and LLaMA-3, which pushed the token-to-parameter ratio even further. LLaMA-3 8B, for example, was trained on approximately 15T tokens, a ratio of nearly 1,875 tokens per parameter. This goes well beyond Chinchilla's 20:1 recommendation, suggesting that for inference-optimized models (where you want the smallest model that meets a performance bar), overtrained smaller models can be the right strategy. Kaplan's framework has no mechanism for recommending this kind of allocation.
Where Both Frameworks Show Cracks
Neither set of scaling laws is a universal law of physics. Both were derived from specific experimental conditions: particular architectures (decoder-only Transformers), particular data distributions (web text), and particular loss metrics (cross-entropy on next-token prediction). Several important caveats have emerged as the field has matured.
First, loss scaling curves do not always translate to downstream task performance. A model can achieve a lower validation loss and still underperform on reasoning, coding, or factual accuracy benchmarks. The relationship between FLOPs scaling laws and actual production utility is noisier than either framework suggests. Second, data quality confounds both predictions. Chinchilla assumed iid data sampling, but real training pipelines involve aggressive deduplication, filtering, and mixing strategies that change the effective information density per token. Training on 1.4T high-quality tokens is not the same as training on 1.4T randomly scraped web text. Third, recent work suggests that scaling behavior may shift at very large scales in ways neither power law cleanly predicts. Emergent capabilities, phase transitions, and the impact of architectural innovations like mixture-of-experts add dimensions that simple parametric fits cannot capture.
Conclusion
The evidence overwhelmingly favours the Chinchilla scaling laws as the more reliable default for planning compute allocation in large language model training. Kaplan's framework overweights parameter count at the expense of training data, a bias that has been repeatedly contradicted by empirical training runs from DeepMind, Meta, and others. That said, neither framework should be treated as gospel. Real-world decisions about optimal scaling depend on data quality, inference cost targets, architecture choices, and downstream task requirements that no single power-law fit can fully account for. For practitioners navigating these tradeoffs, the Chinchilla framework is the better starting point, but the smartest teams treat it as a baseline to iterate on rather than a fixed recipe.
Explore NinjaStudio.ai for technically grounded analysis that cuts through AI hype and helps you make better decisions about model training, deployment, and research strategy.
Frequently Asked Questions (FAQs)
What are scaling laws in AI?
Scaling laws in AI are empirical relationships, typically expressed as power laws, that describe how a model's performance (such as loss) changes predictably as you increase compute, data, or parameter count.
What is the Chinchilla scaling law?
The Chinchilla scaling law, introduced by DeepMind in 2022, states that for compute-optimal training, the number of training tokens should scale roughly linearly with model parameters at a ratio of approximately 20 tokens per parameter.
Can scaling laws predict model performance?
Scaling laws can predict aggregate loss on held-out data with reasonable accuracy, but they are less reliable at predicting performance on specific downstream tasks like reasoning or code generation.
Is there a limit to scaling laws?
Current evidence suggests that scaling laws hold across many orders of magnitude of compute, but data scarcity, diminishing returns on data quality, and architectural shifts may impose practical limits that are not yet fully characterized.
Chinchilla vs Kaplan scaling laws: which is more accurate?
Chinchilla's scaling laws have proven more accurate in practice, as multiple large-scale training runs from leading labs have confirmed that Kaplan's framework systematically underweights the importance of training data relative to model size.