Learning Mechanics

Site TODOs:
   - Redo blurbs
   - Link to whitepaper
   - Write blogpost version of whitepaper for cross-posting in other places
   - Add blogposts from those who've agreed to write/add em
   - Finish quickstart guide, or as much of it as we deem worth finishing
   - Pass through open questions
   - Make 2-3 educational widgets
   - Make site catchy, engaging, fun, visually appealing
   - Add favicon
   - Add mystery button???

Understanding deep learning will be the intellectual challenge of the early 21st century, like physics was in the early 20th. A good place to start is the study of neural network training dynamics, since these give rise to everything else. The study of learning mechanics is rapidly developing into a proper subfield of science. This website is meant to serve as a hub, a place to share progress and organize the search for simple, useful understanding.


A big list of open questions can be found here.

Model your model: a path to verifying theories of deep learning
Enric Boix-Adsera
March 20, 2026
On neural scaling and the quanta hypothesis
Eric Michaud
March 20, 2026
Interactive Tutorials
The Learning Mechanics Team
March 20, 2026
Lazy/Rich Tutorial Deep Linear Nets Tutorial Progressive Sharpening Tutorial KRR Generalization Tutorial
Quickstart Guide: Neuron-level sparsity
The Learning Mechanics Team
September 01, 2025
1. Introduction: asking a specific question 2. The average size of hidden representations 3. Hyperparameter selection (and why theorists should care) 4. The dynamics of optimization 5. 🚧 Feature learning and the final network weights 6. 🚧 Generalization 7. 🚧 Neuron-level sparsity 8. 🚧 The structure in the data 9. 🚧 Places to make a difference
Notes on the Science of Deep Learning
Jamie Simon, Florentin Guth, Dhruva Karkada
August 30, 2025
🚧 Understanding deep learning will pay off in the long run The scientific method in two steps and its application to deep learning 🚧 Science as mapmaking