On Sept. 17, 2026, the National Institute of Standards and Technology’s Center for AI Standards and Innovation published an assessment of Z.ai’s GLM-5.3. CAISI reported that the model was the most cyber-capable open-weight model released to date within the models it compared. It also reported that GLM-5.3’s capabilities were significantly below those of current U.S. frontier models and approximately four months behind them on CAISI’s aggregate capability measure.
What CAISI measured
The assessment evaluated GLM-5.3 on four benchmarks covering vulnerability discovery and exploit development. CAISI’s table lists the model at 40.4% on SEC-Bench Pro, 61.1% on ExploitBench, 9.4% on ExploitGym and 7.7% on CAISI OSS-Fuzz. The listed U.S. frontier-best scores were 90.2%, 100.0%, 44.4% and 23.2%, respectively.
Those figures describe performance on specified benchmark tasks. They do not, by themselves, establish how GLM-5.3 would perform on an arbitrary reader’s codebase or in direct real-world offensive operations.
For the comparison, CAISI defined the U.S. frontier-best result on each benchmark as the highest score achieved by a model released by a U.S. entity that CAISI had evaluated. The comparison excluded models that had been developed but not released. “Frontier” in this assessment therefore names a defined set of evaluated and released systems, rather than every potentially capable U.S. model or every system under development.
CAISI tested the models as tool-using agents in a ReAct harness with access to Bash and Python. It used benchmark-specific turn limits and maximum reasoning settings; when applicable, cyber safeguards were disabled for the U.S. models. These conditions are part of what the scores record. The phrase “four months behind” refers to CAISI’s aggregate capability measure, not simply to a calendar gap between product-release dates.
Why this enters the Record
The significance of the assessment lies in the comparison it makes visible. Its reported result places an open-weight model at the top of a defined open-weight set while also placing that model well below the U.S. frontier as CAISI measured it. Taken together, those findings resist two opposite simplifications: treating the open-weight result as parity with the frontier, or treating the gap as evidence that open-weight systems lack substantial cyber capability.
The assessment also preserves the conditions needed to interpret such a comparison. Model release status, benchmark selection, agent harness, turn limits, reasoning settings and safeguards all shape what the resulting numbers mean. For a future account of AI capability measurement, those methodological boundaries are part of the event itself—not secondary details to be separated from the headline.
An independent analysis by P.K. Sharma reviewed the CAISI assessment and reproduced its central GLM-5.3 and U.S. frontier figures. It explicitly rejected an interpretation that GLM-5.3 had matched the U.S. frontier and noted that benchmark performance should not be confused with performance on a reader’s codebase or real-world operations. Sharma did not independently rerun the evaluation, so the analysis provides independent scrutiny of the reported comparison rather than a replication of the underlying tests.
What remains unknown
The published results do not establish GLM-5.3’s performance outside the specified benchmarks, the model’s direct effectiveness in real-world operations, or the standing of unreleased systems and models that CAISI did not evaluate. They establish a bounded result: how GLM-5.3 scored under the reported conditions and how those scores compared with CAISI’s selected U.S. frontier-best results. That boundedness is central to what the assessment can—and cannot—show.