On September 30, a research team at Z.ai posted a number that should have stopped the crypto industry cold: 4,249 potential vulnerabilities, found across 389 open-source projects. No press conference. No CVE registry entry. No proof-of-concept code. Just a number, a free service called OpenVuln, and a quiet promise that the maintainers had already been notified in private.
I have spent eighteen years watching this space, and I have learned to read the silence around an announcement as carefully as the announcement itself. Crypto built its entire identity on four words: don't trust, verify. We tattooed them onto our whitepapers. We engineered bridges, rollups, and zero-knowledge proofs around them. And yet here is a claim about the safety of our code β the very foundation of every DeFi protocol, every smart contract, every trustless system we have staked our reputations on β that we are simply asked to believe. Trust is no longer a promise; it's a protocol. But a protocol that cannot be audited is just a promise wearing better clothes.
In the summer of 2020, I organized a meetup series in Stockholm called Yield & Connect. We talked about liquidity pools, but the conversation always drifted toward the same anxiety: who actually checks the code? That anxiety has never gone away. It has only gotten more expensive.
Smart contract auditing is the unglamorous backbone of DeFi. Before a protocol can hold a billion dollars, a small team of human experts reads its code line by line, hunting for the logic errors that let attackers drain funds. The work is slow, adversarial, and brutally costly. A serious audit from a top firm β Trail of Bits, OpenZeppelin, Spearbit β can run from $50,000 well into seven figures and take weeks or months to complete. In a bull market, that is a rounding error. In a bear market, it is a line item that gets deferred, and deferred audits are how bridges get drained.
The results speak in blood. Ronin, $625 million. Wormhole, $326 million. Euler Finance, $197 million. The Vyper compiler reentrancy flaw that struck Curve and several other pools in 2023. These were not exotic zero-days pulled from a nation-state's playbook. Most were the same handful of vulnerability classes auditors have documented for years β reentrancy, access control failures, oracle manipulation, unchecked external calls. The bugs are known. The problem is coverage. There are simply not enough human eyes.
That gap is where the AI industry has been circling for two years. Google's Big Sleep project, built on Gemini, demonstrated that a large language model could perform variant analysis β tracing a known bug pattern across an entire codebase to find its siblings. XBOW, an autonomous penetration-testing startup, topped HackerOne's US leaderboard in 2025 by letting an agent hunt real bugs on live targets. OpenAI's Aardvark positioned itself as an autonomous security researcher with publicly documented finds. Each made the same bet: that a model with a long enough context window and the right tooling can reason about code the way a human auditor does, only faster and without getting tired.
Now Z.ai, the lab behind the GLM family of models, says it has done the same thing at a scale none of them have matched. Its tool, OpenVuln, scanned 389 projects and surfaced 4,249 potential vulnerabilities. The pitch is seductive: free security for the open-source commons, powered by a model that reads code better than you do. But I have learned to stop preaching and start listening. And when I listen to this announcement, what I hear is not a breakthrough. I hear a number with no denominator.
Here is the first thing any working auditor notices: the word potential.
In vulnerability research, there are two distinct jobs. The first is candidate discovery β scanning code for patterns that might be dangerous. The second is validation β proving that a candidate is actually reachable, exploitable, and consequential. Candidate discovery is cheap and noisy. Validation is expensive and rare. The entire value of a security researcher lives in the second job.
When Z.ai says it identified potential vulnerabilities, it is describing the first job. The hard part β confirming that a flagged line can actually be triggered by an attacker, building a proof-of-concept, proving the bug matters β is not described at all. Was it done by the model? By a human reviewer? By no one? The announcement does not say. And that silence is not a detail. It is the whole question.
Consider the arithmetic. 4,249 vulnerabilities divided by 389 projects is roughly 10.9 findings per repository. For a mature static analysis tool run against a well-maintained public codebase, that number is not just high. It is anomalous. Commercial SAST tools typically report a handful of high-confidence findings and bury the rest in a triage queue. A rate of nearly eleven per project strongly suggests an unfiltered candidate list: the raw output before anyone asked whether a given finding actually matters.
The false positive problem is not a footnote. It is the headline. LLM-based code analysis, by every published benchmark I have seen, produces false positive rates that would be disqualifying in a traditional security workflow. The commonly cited range for raw model output, before human or automated filtering, sits somewhere between 5% and 30% precision. That means for every real bug, a reviewer might wade through three to nineteen ghosts. Apply that to 4,249 findings, and you are potentially asking unpaid maintainers to investigate thousands of reports, many of which may be hallucinations dressed in convincing syntax.
Now let me be fair to the engineering. The underlying approach is real, and it is not trivial.
The reason LLMs can do this at all comes down to context length and program reasoning. A model that can hold an entire repository in its context window can trace how a value flows from an untrusted input, through several function calls, into a dangerous sink β the kind of cross-file dataflow analysis that traditional tools approximate with brittle rules and that human auditors do by hand. Big Sleep proved this worked for variant analysis. The GLM family is a long-context architecture, so the capability is plausible.
But here is what the announcement conveniently blurs: the model is probably not doing this alone. Serious AI code auditing today is an agentic system β a model orchestrating a chain of tools: code search, static analyzers, fuzzers, symbolic execution engines, and retrieval systems that feed relevant snippets back into the context. The intelligence is distributed across the pipeline, not concentrated in the model. The headline credits GLM-5.3 for protecting 389 projects, but the actual work, if it was done well, was a collaboration between the model and a substantial engineering stack that went unmentioned.
I will be honest about my own blind spot here. My knowledge runs through the GLM-4.5 and 4.6 generations. I cannot independently verify what GLM-5.3 is, whether it shipped, or what it can do. I will treat it as a plausible next-generation model and hold my judgment on the specifics. That uncertainty cuts both ways. It means I cannot dismiss the claim, but it also means the claim cannot lean on my confidence.
This is where the story stops being about open-source software in general and becomes about us.
The projects Z.ai scanned are, presumably, ordinary software β libraries, tools, applications. But the technique's true commercial and ethical weight lands on smart contracts. DeFi is the single most audit-dependent industry on earth. Every dollar locked in a lending pool, every share of a liquidity position, every bridge reserve is governed by code that is public, immutable, and directly attached to money. When a smart contract has a bug, the bug is not a crash. It is a robbery.
And the vulnerability classes are brutally consistent. Reentrancy. Access control mistakes. Oracle price manipulation. Unchecked return values. Signature replay. Integer rounding errors that compound in ways only a patient attacker notices. Based on my years reviewing smart contract disclosures and sitting through more post-mortems than I care to count, I can tell you that a human auditor learns to pattern-match these the way a doctor learns symptoms. An AI system, given enough context, could theoretically scan thousands of contracts for these patterns in the time it takes a human to review one.
That is the promise, and it is a genuinely good one. But it carries a structural problem the announcement never touches: if an AI can find a vulnerability in a smart contract, the same AI can find a vulnerability in a smart contract for the purpose of exploiting it. The tool does not know its user's intent. A scanner that surfaces a reentrancy path for a white-hat audit surfaces the exact same path for an attacker preparing a flash-loan strike. The capability is symmetric. The ethics are not.
The announcement describes only the defensive half. There is no mention of safeguards, no discussion of whether OpenVuln refuses to generate exploitation paths, no acknowledgment that the same research could arm the other side. For a technique this powerful, that omission is not naive. It is a choice.
Let me return to the thing that bothers me most.
In responsible disclosure, the gold standard is the CVE β a public, standardized record that a vulnerability existed, was confirmed, and was fixed. It is not glamorous, but it is the shared ledger of security work. It is how credit is assigned, how trust is built, and how the rest of us know a claimed fix actually happened.
Z.ai's approach produces none of this. No CVEs. No proof-of-concept code. No third-party reproduction. The vulnerabilities were reported privately to maintainers, and then, as far as the public record shows, they vanished. We have a number, and we have a promise.
I understand the reasoning. Coordinated vulnerability disclosure exists precisely to give maintainers time to patch before attackers learn the details. Announcing a bug publicly before a fix ships is how you get a Wormhole. So the private approach is defensible, and in isolation I would call it responsible. But here is the uncomfortable synthesis: the same privacy that protects maintainers also protects the claim from scrutiny. By never publishing a single confirmed finding, never naming a CVE, never releasing a benchmark, Z.ai has created a contribution that cannot be audited. And in a space whose foundational creed is don't trust, verify, an unauditable contribution is a contradiction in terms.
I have watched this pattern before. In 2020, I wrote a piece arguing that DeFi was a protest movement β that its value was social, not merely financial. Critics said I was romanticizing. Maybe. But the reason that argument resonated was that it was falsifiable. People could point to the pools, the communities, the actual users. A number without a CVE is the opposite. It asks for belief without offering evidence, and it dresses the ask in the language of public good.
There is a quieter technical problem, and it determines whether OpenVuln survives contact with reality: cost.
Cross-repository code auditing requires long context. To trace a dataflow across files, a model needs to hold a lot of code in memory at once, often 128,000 tokens or more. And here is the cruel part of the math: inference cost on long-context models scales roughly quadratically with context length. Double the context, and you do not double the cost. You multiply it. Scanning 389 projects, some of them large, is not a rounding error on a GPU bill. It is a serious capital expenditure, repeated every time a repository changes.
Z.ai is offering this for free. Free is a strategy, not a business model. The question any operator should ask is: what is the inference cost per project, and who is paying it? If the answer is Z.ai's investors, for now, then OpenVuln is a marketing expense β a very expensive one β and it will live or die by whether it converts developers into GLM API customers. The announcement does not mention pricing, service-level agreements, or enterprise tiers. That absence tells you what this really is: a developer-relations play dressed as a public utility.
And there is a data angle the announcement skips entirely. A free scanning service that ingests thousands of real codebases is also a data flywheel. Every repository scanned, every vulnerability flagged, every maintainer response becomes training signal. That is not necessarily sinister β it is how modern models improve β but it means the free service has a second product: the labeled vulnerability data itself. Users should know which product they are.
There is also a legal layer left untouched, and it matters enormously for anyone building in DeFi. When a scanning service analyzes a repository, that code has to go somewhere. It gets uploaded, tokenized, processed on servers that belong to someone. Open-source licenses vary; some carry copyleft provisions or restrictions on commercial processing. And once code crosses a border, a new set of rules applies. China's Data Security Law and its cross-border transfer requirements are not trivial, and a repository maintained in Europe or the United States, scanned by a service hosted in Beijing, sits at the intersection of two regimes that do not always agree. None of this is mentioned β not the consent model, not the retention policy, not where the code lives after the scan. You cannot claim to protect a codebase while being vague about what you do with it.
This connects to something I have argued since 2026, when I founded the Human-Centric Blockchain initiative and wrote a manifesto called The Soul of the Code. My thesis then was simple: blockchain's deepest value is not that it moves money without permission, but that it verifies human intent β that it lets us prove what someone actually meant, not just what they signed. AI security tools inherit that same responsibility. A system that scans code without transparency about how it handles that code is verifying nothing about its own intent. The tool becomes a black box guarding a black box.
It is worth placing this against the field, because the scale narrative only works if you do not look sideways.
Google Big Sleep is the reference point, and it earned that status the hard way: it found real, high-severity vulnerabilities in widely used software like SQLite, and the findings were confirmed. Fewer bugs, but each one verified. XBOW competed on a public leaderboard, where its results could be checked and ranked. OpenAI's Aardvark published concrete case studies. The pattern across all three is identical: they competed on verified quality, not raw volume.
Z.ai is competing on volume. 389 projects. 4,249 vulnerabilities. Big, impressive, and, critically, unverified. When precision is unknown, scale becomes a substitute for proof. It is a familiar move in this industry. I have spent years arguing that liquidity fragmentation is a manufactured problem that exists mainly to sell new products, and the dynamic here is the same shape: a metric chosen because it looks impressive, not because it measures what matters.
There is a domestic dimension the international audience misses. Z.ai is not China's only player in AI security β Alibaba, Tencent, and 360 all have competing efforts. The race for credibility in AI-driven vulnerability discovery is being run on multiple tracks, and the winner will be whoever first produces a verifiable, high-severity, publicly confirmed find. Not whoever posts the biggest number.
In a bear market, this matters more, not less. When TVL is shrinking and treasuries are thin, the temptation to swap a $200,000 human audit for a free AI scan is enormous. I have watched protocols make that trade β defer the audit, ship the feature, hope for the best. Sometimes they survive. Often they become a case study. The danger of an unverified AI audit is not that it is wrong. It is that it is cheap enough to replace the thing that was right. A free scan that produces noise can crowd out a paid audit that produces certainty, and the ecosystem ends up less safe while feeling more covered.
Here is the part that unsettles me more than the unverifiable number: the biggest threat from AI vulnerability scanning may not be missed bugs. It may be found ones β too many of them.
Open-source maintainers are already drowning. Most are unpaid volunteers maintaining critical infrastructure in their spare time. Now imagine their inboxes flooded with thousands of AI-generated reports, each technically plausible, most of them false, none accompanied by a proof-of-concept that would let a maintainer triage quickly. The result is not better security. It is notification fatigue β a slow-motion denial of service against the people who actually keep the code safe.
Security is not the absence of reported vulnerabilities. It is the presence of trusted signal. A tool that reports everything reports nothing. And a tool that reports privately, unverifiably, and at scale does not just fail to help. It actively degrades the ecosystem's ability to tell a real threat from noise. This is why I keep coming back to the human layer. Trustless systems require trusting relationships. The code can be trustless. The process of securing it cannot. And the AI industry, in its rush to automate the auditor, is quietly automating away the very accountability that made audits trustworthy in the first place.
So what should you actually watch, if you hold assets in DeFi or build on it?
Not the number. Watch for the first public CVE with Z.ai's name in the credit line. Watch for a benchmark. Watch for a maintainer who confirms a real, high-severity fix. In the next six months, one verifiable finding is worth more than four thousand unverified ones. The AI audit wave is coming for our code whether we ask for it or not. The only question that matters is whether it arrives as a tool we can check, or a promise we can only believe. Code is law, but empathy is the interface. And right now, the interface is asking us to trust it. Verify first.