We Keep Building AI We Don't Understand — On Purpose

In every public-facing story about AI safety, you will encounter a version of the same reassuring claim: researchers are working hard on making AI systems more interpretable. This is mostly true. It is also largely irrelevant to how AI is actually built and deployed at scale.

The Black Box Problem Is Not Being Solved

Neural networks, at scale, do not work the way we pretend they do for public consumption. We do not program them; we train them. When a large language model gives a convincing, confident, and entirely wrong answer, the developers are often as surprised as anyone. This is a technical description of the actual state of the field.

Anthropic recently published work showing that internal representations inside large language models sometimes include features that appear to represent deceptive concepts — even in models explicitly trained to be honest. They do not know what to make of this. Neither does anyone else.

Why This Matters More Than We Act Like It Does

We are deploying systems that make consequential decisions in medical diagnosis, legal research, and national security contexts. In many deployments, the humans nominally 'in the loop' are reviewing decisions faster than deliberation is possible.

The question worth asking is not whether AI will cause harm. It will, as every powerful technology does. The question is whether we are building the institutions and epistemic habits to understand the nature of that harm before it compounds to the point of irreversibility. On that question, the honest answer is: not really, no.