-
LLMs Are Becoming More Similar—to Each Other and to Us
Recently, I read this paper “The Platonic Representation Hypothesis” (Huh et al. 2024). The hypothesis states that all deep neural networks–no matter how they are trained, what training dataset they use, or which modalities they operate on–are converging to one shared statistical model of reality in their representation spaces. This hypothesis is powerful in that Continue reading
-
Is Math Invented or Discovered
There is a classical question that is quite entertaining: is mathematics invented or discovered? Most people (non-mathematicians) tend to think that math is invented, that is the outcome of the work of mathematicians. But mathematicians themselves often express the feeling that they are not inventing math but discovering what is already there. Recently, I read Continue reading
-
How “System 2” Is Born
0. TL;DR Rather than treating System 1 and System 2 as distinct cognitive systems, I argue that “System 2” describes a behavioral profile associated with early-stage use of structured mental tools. I suggest this reframing may be useful for thinking about how reasoning-like behaviors emerge in language models during training. 1. The Classic Two-System View Continue reading
-
Reading Open Problems in Mechanistic Interpretability: A Call for a Complex-Systems Perspective
The recent review Open Problems in Mechanistic Interpretability (arXiv:2501.16496) is one of the clearest snapshots of where the field of mechanistic interpretability (MI) stands today. It’s thorough, honest about its limitations, and refreshingly forward-looking. But while the review is excellent on its own terms, reading it also made something else crystal clear to me: Many Continue reading
-
How a Kid Makes a Rubik’s Cube From Scratch
A Supervisor’s Perspective It started with a bet. Zimo had been watching too many YouTube videos. I was trying to teach him a lesson: “It’s easy when you just watch, totally different when you actually do it.” So I made a 1:100 bet with him. If he could make a Rubik’s cube (because he had Continue reading
-
ESP32 Notes
This post is for documational purposes only. I’ll record some of my ESP32 experiences. (1) When using PlatformIO to flash, pio run -t upload is actually doing this: And boot_app0.bin can be found in ~\.platformio\packages\framework-arduinoespressif32\tools\partitions\boot_app0.bin (2) When using littlefs, we can’t mount a partition to / because the library will check the last char, if Continue reading
-
Embodiment and Abstraction in Artificial Intelligence: Building the Skyscraper of Intelligence
In recent years, embodiment has moved from the margins to the mainstream in artificial intelligence (AI), gaining traction in both academia and industry. Once a niche interest — championed by philosophers, enactivists, and a handful of forward-looking scientists — it is now widely seen as essential to the future of intelligent systems. Many argue that Continue reading
-
Religions and Science
In the beginning, human beings tried to make sense of the world around them. They observed, guessed, discussed, and formed explanations for the things they didn’t understand–like the stars, the weather, life, and death. These early efforts became what we now call religions. They weren’t just belief systems; they were thoughtful attempts to understand the Continue reading
-
Is Wolfram’s Computational Irreducibility Too Strong?
Stephen Wolfram introduced the idea of Computational Irreducibility, which says that some systems are so complex, there’s no shortcut to predict what they’ll do–you have to go step by step, like running a full simulation. He suggests that in these cases, there’s no way to simplify the process. I love this idea, but if we Continue reading
-
Reading A Universe of Consciousness
I’ve just finished reading A Universe of Consciousness by Gerald Edelman and Giulio Tononi. Rather than focusing heavily on experimental data, the book offers a high-level conceptual framework for how the brain gives rise to consciousness. That approach resonates with me—once we understand the overarching structure, we can begin to interpret experimental findings as either Continue reading
Sida Liu 