You Cannot Automate a Fix You Cannot Trust

Third in a series on the throughput gap.

I have spent the last two years on a problem that sounds narrower than it is: how do you know a security finding is real before you act on it?

Everything else in autonomous defense depends on the answer. Galina wrote about the throughput gap - the distance between how fast risk arrives and how fast defenders close it. Eran wrote about what it takes before a customer will let a machine close anything. Both of those come back to the same engineering question, and it is not a question about models.

Here is the thing I would want a technical reader to take away. The ceiling on remediation throughput was never reasoning capacity. It was confidence. A security team can only act as fast as it can be sure. Everything slow about vulnerability management - the triage, the back-and-forth with asset owners, the tickets that sit open for 43 days - is the cost of manufacturing certainty with manual analysis.

So, if you want to raise the ceiling, you do not start by making the fix faster. You start by making certainty cheaper.

Why raw data does not go to the model

There is a design decision at the center of Kai that I get argued with about, so let me explain the reasoning rather than just assert it.

Raw security data is a mess. Scanners disagree. The same vulnerability appears under four identities across three tools!

Asset inventories are stale. In the environments we work in, CMDBs are somewhere between 50-60% complete on ownership, and you cannot remediate what you cannot attribute. In short, findings arrive duplicated, malformed, and without context at volumes in the hundreds of millions.

The instinct is to point a capable model at that pile and let it sort things out. Models are good at ambiguity, after all.

We do not do that, for two reasons.

The first is quality. A language model given inconsistent inputs will produce a fluent, confident answer built on those inconsistencies. It does not fail loudly. It fails plausibly, which is worse, because plausible failure is what survives review. When the foundation is fragmented, AI makes fragments faster. It does not close the gaps between them.

The second is that it is wasteful. Deduplication, normalization, and validation are deterministic problems with deterministic answers. Solving them with inference is slower, more expensive, and less reliable than solving them with algorithms. You should spend model capacity on the parts of the problem that genuinely require judgment, not on work a well-written function does perfectly every time.

So at the core of the architecture there is a harness: a layer of deterministic algorithms that ingests, cleanses, deduplicates, validates, and chunks raw security data before any of it reaches the reasoning layer. Aithen does the work that actually needs reasoning, resolving inconsistent findings, extracting meaning from unstructured data, and assembling context. We route across model tiers depending on what the task requires because for a platform handling data at this sensitivity the access path to the model is not an implementation detail. It is a security decision in its own right.

That boundary - deterministic before probabilistic - is the whole design. It is also the reason we implemented LLMs as a fully autonomous system rather than as a chatbot. You can put a model inside a sensitive workflow and keep control, auditability, and reliability, but only if you are disciplined about which parts of the workflow it owns.

What validation actually has to establish

“Validated” is a word vendors use loosely. Here is what I mean by it.

A finding is not real because a scanner emitted it. A finding is real when you can establish, in the specific environment, that the vulnerable code path is reachable, that the exposure is exploitable in context rather than in theory, and that it matters given what the asset actually is and does. That means combining static analysis with runtime behavior, reasoning about chaining - because two medium findings that compose into a full compromise are not two mediums - and assessing network reachability rather than assuming it.

The output that matters is not a score. It is definitive proof of exploitability, produced autonomously.

This is the unlock, and I want to be clear about why. Once exploitability is proven, prioritization stops being necessary. Ranking exists because you cannot fix everything, and you need a defensible way to choose. If the false positives are gone and the remainder is confirmed real, the question is no longer which findings to work. It is throughput. Prioritization was always a rationing mechanism for scarce human attention, and rationing is what you do when supply is fixed.

The numbers follow from that. In one deployment, 250 million vulnerabilities from enterprise scanners investigated and triaged in 20 hours, 83% eliminated as benign, the rest auto-remediated. In another, 2.5 million software composition analysis findings across 5,000 container images processed in under an hour, 99.5% eliminated as false positives, roughly 3 million engineering and security hours saved annually. In a third, 10 million infrastructure findings triaged in three and a half hours, 4 million confirmed as real risk, 3.8 million auto-remediated.

None of those numbers are about a model being clever. They are about not asking a model to do work that determinism does better, and then having enough confidence in the output to act on it without a human re-checking each one.

The interval nobody measures

One more piece, because it took a customer to make me see it.

Even with validation solved and remediation automated, some fixes cannot land immediately. A patch needs a maintenance window. An owner needs a day to review a change to a production system, usually taking 10-15 days. Customers told us that deploying mitigating controls in that interval is practically possible only in a handful of cases, because writing and tuning detections by hand is its own slow, specialized job.

That interval is a real exposure window, and almost nobody measures it. So, Kai Watch generates and deploys context-based detection rules the moment an exposure is confirmed, then monitors SIEM, EDR, and XDR telemetry for signs of active exploitation while the fix moves through whatever process it has to move through. It works with the tools already in place rather than replacing them.

It is not a substitute for the fix. It narrows the window in which the unfixed thing is also unwatched.

Why this order matters

If I could put one idea in front of every team building in this space, it would be this: autonomy is downstream of certainty.

Every failure mode people fear about automated remediation - fixing the wrong thing, breaking something, acting outside its authority - traces back to acting on a finding that was never established as real. Build the validation layer first and the automation becomes a reasonable engineering decision. Skip it and no amount of model capability will make the automation safe, because the model was never the weak link.

Prioritization kept the backlog alive. Validated, verified execution is what retires it.