The Verification Gap

An observation from the wrong side of the table

This past year I worked with two of the largest defense primes in the country to get customer case studies written, reviewed, and approved. The work meant sitting with engineers to understand what they had actually built, then walking the resulting content through internal communications, legal, and security review.

Most of what I learned had nothing to do with content approval.

What I learned is that defense engineers do not evaluate claims the way commercial buyers do. A commercial buyer hears "reduces vulnerabilities by 90 percent" and files it as a benefit. A defense engineer hears the same sentence and immediately asks a different question: by what mechanism, measured how, and what happens at the edges where it does not work. Not out of hostility. Out of professional habit. These are people whose work gets audited, whose systems carry consequences, and who have been burned before by tools that tested well and behaved badly.

The second thing I learned is that their leadership was pulling in a different direction. Program leaders wanted their engineering teams adopting new capability faster. They were not confident about how to govern it. The gap between "we should be using this" and "we know how to prove it is safe" was where every difficult conversation lived.

At the time this was a communications problem I had to solve. In 2026, watching AI-generated code move into federal software supply chains, it looks less like an anecdote and more like the central dynamic of the market.

This is an attempt to lay out why.

The trust gap is real, measurable, and moving in the wrong direction

The most useful thing about the current moment is that the intuition is no longer anecdotal. Multiple independent surveys have measured it.

Adoption is close to universal. Stack Overflow's developer survey, drawing on roughly 49,000 responses, found that around 84% of developers use or plan to use AI coding tools. Sonar's survey of professional developers found that among developers who have tried the tools, most now use them daily, and they estimate roughly 42% of the code they commit is AI-assisted.

Trust moved the opposite way. Only 29% of Stack Overflow's 2025 respondents said they trusted AI output, down eleven points from the prior year. Forty-six percent actively distrust what the tools produce, and only about 3% describe themselves as highly trusting — a figure that drops further among experienced developers. Sonar's number is framed differently but points the same direction: 96% of developers report difficulty trusting that AI-generated code is functionally correct.

A note on those figures, because precision matters to the audience this concerns. The surveys do not agree exactly. Trust in accuracy has been reported at 29% and at 33% depending on the instrument and the phrasing; Sonar's 96% measures something narrower than general distrust. The disagreement is not a reason to discount the finding. Every instrument, run by different organizations with different incentives, moved the same direction over the same period. Treat the direction as solid and the decimal places as contested. Figures throughout this section are drawn from the Stack Overflow Developer Survey 2025 (survey.stackoverflow.co/2025, n≈49,000) and Sonar's State of Code developer survey (n=1,149 professional developers). Where secondary reporting of these surveys conflicts, I have deferred to the published instruments.

The most telling number is not about trust at all. It is about behavior. Despite that widespread doubt, only about 48% of developers say they always check AI-generated code before committing it. Sonar's own framing of the cause is worth taking seriously: reviewing AI-written code is genuinely hard work, and the tools that were supposed to reduce effort have partly relocated it downstream.

And the reason review is hard is specific. The dominant complaint is not that AI produces garbage. The top frustration, cited by 66% of respondents, is AI output that is almost right but not quite. Broken code announces itself. Plausible code does not. Engineers are being asked to spend attention on a failure mode that is optimized to survive casual inspection.

That is the shape of the problem: near-universal adoption, declining confidence, and a verification step that does not scale with generation volume.

Why federal is a different problem than enterprise

An enterprise engineering organization experiencing this dynamic has a bad quarter. A federal program experiencing it has an authorization problem, and the difference is structural.

Consequence attaches to documentation, not just to outcomes

In federal software, security posture is not merely a state of the system — it is an evidentiary claim made to an authorizing official, in a package, that someone signs. Authority to Operate is granted on the strength of that package. When a meaningful and growing fraction of the code in a system was generated by a model, the honest version of the evidentiary claim gets harder to write, because the traditional chain of reasoning ("a qualified engineer wrote this, another reviewed it, here is the test coverage") has a link in it that behaves differently than it used to.

The governance layer arrived faster than the trust layer

Federal AI policy has moved with unusual speed. Section 1512 of the FY2026 NDAA directs the Department of Defense (now the Department of War) to establish a department-wide cybersecurity and governance policy for all AI and machine learning systems it uses, explicitly addressing data poisoning, jailbreaks, counterfeit parts, and unauthorized access. Section 1513 adds a risk-based framework for cybersecurity and physical-security procurement requirements covering AI systems. Section 1521 is the most directly relevant here: it amends the Department's Authority to Operate processes to include mandatory timelines (Congressional Research Service, IF13197).

On the civilian side, Executive Order 14409, signed June 2, 2026 and published at 91 FR 34565, directed CISA to issue binding operational directives to expedite the cyber defense of civilian federal information systems and to expand AI-enabled defensive tooling. Eight days later CISA issued BOD 26-04, "Prioritizing Security Updates Based on Risk," which compresses the remediation window to three calendar days for vulnerabilities meeting all four of its risk criteria — publicly exposed, listed on the Known Exploited Vulnerabilities catalog, automatable, and granting total control on exploitation. It is the most aggressive standing remediation timeline CISA has issued, and it supersedes BOD 22-01 and BOD 19-02.

The scope is worth stating precisely: binding operational directives bind federal civilian executive branch agencies only — not the Department of War, and not contractors, though prime flow-downs have historically carried them further than their formal reach. CISA has separately published guidance instructing agencies to treat open-source AI systems differently from other open-source software, on the grounds that open source licenses for AI do not require the transparency needed to evaluate trustworthiness.

That last point deserves emphasis, because it is a federal agency stating plainly that the existing trust model does not transfer. The policy apparatus has correctly identified that something is different. It has not yet supplied engineers with a practical method for establishing confidence in the artifact in front of them.

The clock ran out at the same time

The remediation timelines tightening across federal guidance are a response to adversarial acceleration. CISA states in BOD 26-04 itself that threat actors' use of AI may narrow the time defenders have between a patch being released and exploitation. The gap this opens is measurable: citing the 2026 Verizon Data Breach Investigations Report, CISA notes that only 26% of catalogued known-exploited vulnerabilities were fully remediated in 2025, down from 38% the prior year, against a median resolution time of 43 days. A 43-day median against a three-day requirement is not a gap that closes by adding staff. So the same environment now demands both faster remediation and more rigorous verification of code that engineers are less confident in than they were two years ago. Those two demands are in direct tension, and the tension lands on the same people.

What SBOMs answer, and what they do not

The reflexive answer to "we don't know what's in our software" has been the Software Bill of Materials, and it has been a genuine improvement. It is now standard practice, mandated in federal contexts, and it solved a real problem: nobody could enumerate their dependencies.

But it is worth being precise about what an inventory can and cannot establish, because AI-generated code exposes the boundary.

An SBOM answers what components are present. It does not answer what executes. This distinction predates AI and is well understood by practitioners — a container image can carry hundreds of packages, most of which never run in production, each contributing CVEs — Common Vulnerabilities and Exposures, the publicly catalogued software flaws that carry identifiers in the form CVE-YYYY-NNNNN — to a report that consumes analyst attention without corresponding to real exposure. Several vendors have built approaches around runtime observation precisely to separate theoretical inventory from actual attack surface, and the underlying logic is sound regardless of whose product implements it.

More relevantly here: an SBOM tells you a component's provenance, but provenance is not the question AI raises. When a human engineer writes a function, the trust chain runs through the engineer's judgment, their team's review, and the organization's process. When a model writes it, the artifact may be indistinguishable in the bill of materials — it is simply first-party code — while the trust chain underneath it has quietly changed shape.

This suggests something uncomfortable but, I think, correct: the AI code problem may not be solvable at the inventory layer at all. Attempting to solve it there produces proposals for AI provenance labeling — marking which code a model wrote — which sound satisfying and are of limited practical use. Knowing that a model generated a function does not tell you whether it is correct. Knowing a human wrote it does not either. The label answers an authorship question when the operative question is behavioral. I hold this position knowing it is contested; there is a reasonable counterargument that labeling, whatever it fails to establish about correctness, is a precondition for enforcing policy at all.

Runtime evidence is at least the right category of answer, because it is indifferent to authorship. What actually executes, what it touches, what it never invokes — those observations are equally valid whether the code came from a senior engineer, a contractor, or a model, and they are the observations that map to attack surface rather than to inventory.

I want to be careful not to overstate this. Runtime observation is not a complete answer to AI code risk; it says nothing about logical correctness, and a functionally wrong routine that runs is still wrong. The claim is narrower: for the specific question of exposure, behavior is a more durable evidentiary basis than authorship, and it does not degrade as the authorship mix changes.

The part that is not a tooling problem

Here is where the observation from the primes becomes load-bearing.

Suppose the technical answer is entirely correct — that behavioral evidence is the right foundation for governing AI-assisted code in authorized environments. That would not, on its own, change anything. Because the population being asked to adopt it is the same population currently reporting the lowest trust in AI it has ever reported, inside institutions that have historically responded to uncertainty by adding review rather than removing it.

Stack Overflow's own analysis of the trust gap ("Mind the gap: Closing the AI trust gap for developers," February 2026) is more honest about this than most vendor commentary: it notes that when engineering teams do not trust a tool, narrow pilots may succeed while the tool never scales, and that teams under pressure revert to processes they already trust. That is precisely the failure mode I watched play out with new tooling inside defense programs, years before AI was the topic. A capability that does not survive contact with a skeptical engineer does not get adopted. It gets tolerated during the pilot and abandoned at the first deadline.

Which means the adoption path for anything in this space runs through explanation, not persuasion. Some observations on what that looks like in practice, drawn from watching it succeed and fail:

Mechanism beats outcome

"Reduces attack surface by 90 percent" is a claim an engineer cannot evaluate, so they discount it entirely. "Observes which components are invoked during runtime, removes the ones that never are, and here is what happens when coverage is incomplete" is a claim they can reason about. The second sentence is longer, less quotable, and dramatically more effective with this audience.

Stated limits build more credibility than stated capabilities

The most trust-generating sentence a vendor can write in this market is an accurate description of what their product does not do. I have seen technical audiences move from skeptical to engaged on the strength of a vendor volunteering a limitation before being asked. In a market where compliance-adjacent claims are routinely stretched, honest scoping is a genuine differentiator — and it is currently sitting in most vendors' FAQ pages, phrased as legal hedging, where it reads as evasion rather than as candor.

The two audiences need different documents, and conflating them fails both

The engineer needs mechanism and limits. The program leader needs schedule, authorization risk, and cost of the status quo. Most material in this category is written for the engineer and then handed to the leader, or written for the leader and dismissed by the engineer as marketing. The leader's question — whether this shortens the path to authorization enough to matter — is a legitimate question that deserves its own artifact rather than being backed into from CVE counts.

Education is a longer commitment than a campaign

The organizations that will actually move this are the ones treating engineer trust as something built over quarters through documentation, reproducible demonstrations, and public method — not as something a launch converts. That is an unglamorous answer, and it is the one that matches how these institutions actually change their minds.

What I would watch

A few open questions I do not think anyone has answered well yet, offered without a view on who should answer them.

Does the verification gap close on its own, or does it widen?

If AI-generated share of committed code moves from roughly 42% toward the frequently projected 65%, and verification practice stays where it is, the exposure compounds. The optimistic case is that tooling for automated verification matures fast enough to absorb the volume. The pessimistic case is that organizations quietly redefine "reviewed" downward under schedule pressure. Both are happening somewhere right now.

Do authorizing officials develop a position on AI-assisted code, and what is it?

So far federal AI governance has focused on AI systems — models, agents, their security properties. Comparatively little has addressed conventional software that happens to have been substantially written by a model. That is arguably the larger surface area, and it is currently governed by frameworks that predate the question. Section 1521's ATO timeline amendments are the closest thing to a hook, and they do not address authorship.

Does the honest-scoping approach get rewarded?

My belief is that in this specific market, with this specific audience, accurate description of limits is a competitive advantage rather than a liability. That belief is not universally shared, and the next few years will test it.

The observations here about how technical audiences in defense evaluate claims come from direct experience doing content and communications work with prime contractor teams. The survey data is cited to its sources and, where instruments disagree, the disagreement is noted rather than resolved. Corrections and counterarguments welcome — particularly from people currently inside authorization processes, where I have read the policy but not lived the practice.