This was the year AI safety became a public worry instead of a niche one. Polls showed most Americans felt AI companies weren’t doing enough to prevent serious harm, while companies, their executives, and the White House responded with new proposed safeguards. Nearly all those safeguards work the same way. They ask the public to trust someone, usually the companies themselves or the experts hired to evaluate their work.
The concern has reached beyond technology and politics. In May 2025, Pope Leo XIV identified artificial intelligence as a defining challenge of his papacy, warning that its development must respect human dignity and responsibility. His intervention underscored how AI safety had become a question of public trust and accountability.
&Truth, a public benefit corporation co-founded by AI safety advocate Chris Hsu, Stanford professor Clark Barrett, and National University of Singapore professor Ilya Sergey, is betting on a different kind of safeguard: mathematical proof. A proof can establish, for example, that specified software cannot access the internet under precisely defined assumptions. Others can independently check the proof, without relying solely on the company’s assurances.
AI Safety’s Crisis of Trust
On Sept. 29, six major AI companies, Anthropic, Google, Meta, Nvidia, OpenAI and xAI, signed the voluntary White House Accord on Super Intelligence, committing to internal controls, an independent external evaluator and an independent board committee. Earlier that month Anthropic CEO Dario Amodei proposed embedding outside evaluators inside frontier labs, with badges, desks and access “mostly comparable to what internal risk assessment teams have.” At the same time, Bill Gates told the New York Times’ Ezra Klein, “There is no supervisory layer today.”
Public opinion moved faster than any of it. A Reuters/Ipsos poll of 1,277 adults released Sept. 22 found 73% saying AI companies had not gone far enough to prevent serious harm to society, and 39% saying AI was having a negative effect on society, up from 36% a month earlier and the highest reading since the question was first asked in March.
A Gallup survey conducted with the Special Competitive Studies Project found 79% wanted the government to keep its safety and data-security rules even at the cost of slower AI development. Only 7% put speed first. Three in four wanted new systems tested by independent experts before wide release.
The Limits of Institutional Assurances
Put this year’s proposals side by side: internal controls, independent audits, board oversight and outside evaluators working inside the labs. Each one ends with a person or an institution whose judgment the rest of us are asked to accept. An internal control is a company’s promise that it follows its own rules. An audit is a report most people will never read. An embedded evaluator, however independent, is still one person trying to keep up with AI systems that work far faster than any human can follow. The International AI Safety Report 2026 adds a harder problem: frontier models increasingly behave differently when they detect they are being tested, which means a system that can tell when it’s being evaluated can’t be made safe by evaluation alone.
According to &Truth, these efforts remain essential. But audits, evaluations and governance cannot by themselves establish that software satisfies a security property across every case covered by a formal model. They’re only as believable as whoever is making the claim, and the polls suggest the public’s faith in those institutions is already thin.
There is another kind of evidence, and computer science has had it for decades. The agencies that certify aircraft and high-security software already accept it, yet it has rarely come up in arguments about how to govern AI. It’s mathematical proof. Aviation certification admits formal proof as evidence under DO-333, the formal-methods supplement to DO-178C, and the top Common Criteria assurance level for security software requires formal design verification. The seL4 microkernel has carried a machine-checked proof of correctness since 2009.
Mathematical Proof Changes the Equation
A valid formal proof can be checked independently, regardless of who produced it. That does not eliminate trust altogether: the specification, modeling assumptions and proof-checking kernel must themselves be scrutinized. The checking is done by software called a proof assistant, and Lean is one of the best known. It breaks an argument into small steps and runs each one past a compact core program that experts can inspect on its own. That core checks every step the same way, whether a person or an AI wrote the proof.
For software, that changes what a safety claim means. Take a sandbox, the walled-off space where AI labs run a system so it can’t touch anything outside. A lab that says its sandbox is secure is asking the public to take its word for it. A lab that publishes a machine-checked proof that nothing inside the sandbox can reach the internet, under the conditions the proof spells out, is offering something anyone can verify, including its rivals. The claim no longer depends on who’s making it. Hsu’s extension of this to policy is direct: regulators could require labs to commit to precise constraints and provide checkable proof, so an embedded evaluator points to a proof rather than offering a judgment.
This application to regulatory oversight is just one implication of &Truth’s broader principle: trust in software should rest on independently checkable proof, not any company’s word. That principle is central to &Truth, a public benefit corporation legally required to weigh its public mission alongside profit. Its co-founder, Chris Hsu, has been making that case.
Chris Hsu and &Truth’s Solution
Chris Hsu is Co-Founder, Co-CEO and Executive Chairman of &Truth. He is also a member of the Lean FRO Strategic Advisory Board. Previously, he was Founding Partner and CEO of Kilometre Capital and Managing Director of Citadel Investment Group. Since 2025, before co-founding &Truth, he has argued that formal verification, the work of proving mathematically that software does what it’s supposed to, was an underfunded but vital foundation for AI safety. As President of the Infinitude Foundation, he has given grants to various groups working on AI safety and formal verification. &Truth itself, which he co-founded in early 2026 with Clark Barrett of Stanford and Ilya Sergey of the National University of Singapore, is a public benefit corporation headquartered in Palo Alto and Singapore and funded with $70 million of seed capital from AI safety and venture investors.
Hsu’s argument comes down to the difference between probability and proof. An AI agent might be 90, 95 or even 99% likely to do what it’s supposed to. Hsu argues that the remaining doubt is what keeps people from giving such a system real authority. Would you, he asks, let an AI agent run your bank account on its own if you were 99% sure it would behave? Most people wouldn’t. Mathematical proof offers a different foundation for trust: certainty that software satisfies precisely defined requirements, across all cases covered by the proof.
For decades, proof was considered too expensive to use anywhere except where failure is unthinkable, such as aircraft software and railway signals. That’s changing quickly. Hsu points to early 2026 as the turning point. AI tools began translating code and math into the precise language a proof checker can read, and work that once took teams of experts years started getting done in days, sometimes hours, which is what makes proof practical at scale.
The public record now supports the claim. Anthropic reported on Sept. 4 that Claude had produced a complete, computer-checked Lean proof of Fermat’s Last Theorem in 11 days, and on Oct. 22 that, working with the startup Theorem, it had proved memory-safety properties for 239 fuzzing targets across 123 open-source libraries at the machine-code level, finding bugs along the way that years of testing had missed.
&Truth aims to push this transformation much further. Hsu believes AI has already made formal verification roughly 100 times faster and cheaper than traditional human-intensive methods. &Truth’s near-term ambition is to achieve another 100-fold improvement through autonomous proof automation, potentially making verification 10,000 times faster and cheaper than traditional approaches. The goal is to make mathematical proof practical at the scale of the world’s software.
But a proof covers only the properties someone states. It establishes that code matches a specification, not that the specification captures what anyone intended, which means the burden of human judgment moves upstream rather than disappearing. Hsu has said as much himself, and argues that capturing human intent is now where the hard work sits.
&Truth calls its answer a “specification factory,” tooling to turn human intent and business requirements into precise formal specifications at scale. The division of labor is explicit: deciding what a system must guarantee, and what it must never do, is a human judgment; proof guarantees the system honors that judgment, it doesn’t replace it. The ultimate goal is to ensure that software does exactly what humans intend, not merely what its code happens to instruct.
&Truth is working to bring mathematical proof to the world’s software, across programming languages, applications and entire systems. Building on Lean and open-source foundations such as CSLib, it is tackling a challenge that spans the foundations of modern computing: making verification a standard part of how software is built, rather than a privilege reserved for a few critical systems.
As AI systems gain greater autonomy, the question extends beyond whether software works to whether it reliably serves human intent. Mathematical proof cannot determine what AI should do, but it can establish whether software meets the requirements humans define.
Hsu’s vision of a safe AI future is one in which increasingly autonomous systems remain accountable to human intent, with mathematical proof establishing that software satisfies the requirements humans define. In that future, trust would rest not simply on what companies promise, but on what anyone can independently verify.
