Anthropic CEO Dario Amodei has raised a red flag about how little we understand the inner workings of today’s most advanced AI systems. In a new essay titled The Urgency of Interpretability, Amodei outlines the company’s vision to change that — setting a goal to detect and explain most AI model issues by 2027.
Despite the powerful capabilities of large language models (LLMs), researchers still struggle to explain how they make decisions. This gap, Amodei argues, is not just a research problem — it’s a safety issue. As AI systems play larger roles in the economy, national security, and everyday tech, our lack of insight into their “thinking” could pose serious risks.
“These systems will be absolutely central… and will be capable of so much autonomy,” Amodei wrote. “It’s basically unacceptable for humanity to be totally ignorant of how they work.”
Cracking Open the Black Box of AI
Anthropic is one of the few AI companies heavily focused on mechanistic interpretability — the science of understanding AI decision-making from the inside out. While AI models grow more accurate and powerful, the reasons behind their outputs remain murky.
Amodei notes that breakthroughs are beginning to surface. His team has traced simple reasoning patterns, or “circuits,” in their models. For instance, one discovered circuit helps the AI recognize which U.S. cities belong to which states. But these findings barely scratch the surface — Anthropic believes there are millions of such circuits within any given model.
This mystery isn’t limited to Anthropic. OpenAI recently released models like o3 and o4-mini that perform well on complex tasks. Yet, they also hallucinate more frequently, and the company doesn’t know why. That lack of transparency reflects the broader challenge across the industry.
“When a model summarizes a financial report, we still don’t know why it chooses certain words, or why it slips up, even if it’s usually accurate,” Amodei wrote.
Anthropic cofounder Chris Olah puts it another way: these models are “grown more than built.” Researchers know how to make them smarter — they just don’t fully understand why the improvements work.
Toward Safer, Smarter AI Systems
Amodei believes failing to understand advanced models before reaching AGI (artificial general intelligence) could be dangerous. He compares a future AGI system to “a country of geniuses in a data center.” Without insight into their reasoning, humanity risks losing control over what such systems might do.
To address that, Anthropic wants to build tools that act like “brain scans” or “MRIs” for AI. These diagnostic systems could reveal potential issues — like deception, power-seeking behavior, or flaws in logic — before models are widely deployed. However, Amodei admits these tools might take 5–10 years to develop fully.
Still, progress is happening. Anthropic has made some internal advances and recently backed a startup focused on AI interpretability. While this area is often treated as a safety concern, Amodei argues that clear explanations of AI behavior could also create a strong commercial edge. Companies may prefer models that not only perform well but can justify their actions.
Amodei also calls on other major players like OpenAI and Google DeepMind to step up their interpretability efforts. At the same time, he’s urging governments to support this research with “light-touch” regulations. These could include safety disclosures and transparency standards for AI companies.
In a more pointed proposal, Amodei recommends export controls on advanced chips to China, aiming to prevent an unchecked global race to develop powerful AI systems without proper oversight.
Anthropic’s stance reflects a broader trend: the shift from building smarter AI to building understandable AI. While many companies are racing to push model performance, Anthropic is pressing for deeper insight into what lies under the hood.
As Amodei puts it, the world may be years away from truly understanding these complex machines. But without that understanding, scaling their use could carry real-world risks that no one — not even their creators — can fully predict.
