To the Bottom

Neural networks have an unusually accessible bottom level. In fields such as physics and biology, the fundamental processes underlying observed phenomena are often difficult to access directly. Researchers must develop instruments, design indirect measurements, and infer hidden mechanisms from incomplete observations. Neural networks offer a remarkable opportunity: their elementary computations are almost entirely accessible to us. We can inspect the actual numerical values, mathematical operations, and parameter updates that constitute their behavior and learning. This makes neural networks particularly exciting subjects for mechanistic investigation. We have an unusual opportunity to study a complex system from the bottom up, with direct access to the computations from which its higher-level behavior emerges.
This accessibility invites us to investigate all the way down to the fundamental computations. Rather than stopping at familiar abstractions such as models, layers, or training runs, we should examine what actually happens within them, down to individual numerical values and operations. Abstractions are useful, but they should not become black boxes themselves. We should be willing to investigate details that are usually averaged over, summarized, or ignored, without assuming in advance that they are unimportant.
By examining these numerical details, we may discover structures that higher-level abstractions obscure. Through statistical analysis, visualization, and other computational tools, we can uncover previously overlooked patterns. For example, a collection of 1024-dimensional vectors may appear overwhelmingly complex when treated as opaque objects. But examining their numerical structure might reveal that most of their variation is concentrated along just two or three dominant directions. What initially seemed high-dimensional may have a surprisingly simple underlying structure.
From these patterns, we can build explanations and theories. Once previously hidden structures become visible, simple mechanistic explanations may emerge naturally. We can then formulate hypotheses, develop mathematical theories, and derive predictions. Finally, we can test whether these explanations hold across different models, architectures, tasks, and scales. In this way, theory grows out of concrete observation rather than being imposed on the system in advance.