The paper: Hoffmann et al., "Training Compute-Optimal Large Language Models" (2022) — universally known as Chinchilla.
What it found: for a fixed compute budget, the models of the day were substantially too large and trained on far too little data. The optimal ratio was roughly twenty training tokens per parameter, far more data-heavy than prevailing practice. A smaller model trained on more data beat a larger one trained on less, at equal cost.
Why it mattered
It redirected the field away from parameter count as the headline number. The models that followed were smaller and trained far longer, which also made them cheaper to run — an inference-cost benefit the paper did not emphasise but that has shaped the economics since.
What has changed since
The compute-optimal frontier assumes you care only about training cost. In practice inference dominates lifetime cost for a deployed model, which pushes the optimum further toward smaller models trained past the Chinchilla point. Data quality and repetition effects also complicate the clean ratio.
What still holds: the central correction — that parameter count alone tells you little, and that data volume was badly undervalued — has not been overturned.