Introducing Kai Labs: A New Approach to Security Research

What we are not doing

I feel strongly that it is worth being precise about this. Research labs in our industry have a familiar shape, and we don't quite fit that mold.

We are not a vulnerability research team discovering, reporting, and publishing CVEs. We are not going to reverse engineer malware, and we are not going to name and track threat actors across infrastructure and tooling.

These are difficult disciplines practiced by people and organizations who are very good at them. Our reading lists are full of their work, and we intend to keep following and citing it, because good research is good research, even when the author is also a competitor.

But that ground is well covered, and adding another voice to the mix won't answer the questions I hear most from practitioners on the ground. We're interested in what happens after the vulnerability drops: how to prevent the attack, and how to remediate at scale.

Why this lab, and why now

If you've been to a conference or even just a talk about vulnerability management in the past decade, you've seen the slide about the number of vulnerabilities published per year. You've heard that the volume is increasing, and you've heard that backlogs are growing. There's a new zero day before you've finished reading the bulletin about the last one.

Maybe you've felt this recent increase. Maybe 2024, or 2025, or 2026 has been uniquely rough for you. But I often hear from practitioners who were at the edge of capacity before this latest spike. This isn't just increasing the pressure. It's pushing teams into crisis and burnout even as the tide continues to rise.

The predictable response, one we're seeing play out across the industry, is to push defensive work toward not only automation but autonomy. I think this is the right direction. I also think it's headed toward a wall that has little to do with machine capability.

The wall is a lack of trust. We're still proposing "human in the loop" as a solution to hesitance, and now teams are being asked to implement full autonomy. They're right to be skeptical; I share their skepticism. A model being impressive simply isn't enough. It must behave as expected when expected and no more, and it must not compromise the reliability of the existing infrastructure as it acts.

Attackers (usually) don't care what they wreck on the way to their goal. The subgoal of avoiding detection sometimes coincides with limiting the blast radius, so to speak, but they aren't typically concerned with collateral damage. Defenders, on the other hand, must protect their systems without impacting operations, even if that leaves attack vectors open. Not using email would certainly keep employees from falling for phishing, for example, but you won't hear that idea proposed after a compromise.

The gut hesitation I often see when autonomous defense comes up speaks, for me, to the fact that defenders don't trust autonomous systems not to cause collateral damage. Automated network isolation in response to an anomaly sounds fantastic, but what if it doesn't trigger when it should? What if it does trigger when it shouldn't?

That uncertainty comes from a lack of data. Without specifics, customers are left to fill in the blanks. What they need are detailed metrics and, ideally, a confusion matrix. With those metrics and knowledge about the data, customers can calculate their own threshold for acceptability. Rather than a vendor trying to put a number on the risk their tool may introduce, customers can estimate the cost of a mistake from their own internal data.

The situation we're in is not an untenable one, but vendors saying "just trust me" isn't a fix. At Kai Labs, we aim to equip practitioners with the data they need to trust truly innovative solutions. If we can give defenders trust in the solutions they're offered, if we can give them data to share internally, if we can talk openly about limitations, that is how we empower teams to make real change.

For autonomous defense to be widely adopted, or adopted at all, it must not introduce more risk than it resolves. And yet that's where we find ourselves now: in a situation where the answer to "will this system cause problems?" is "I can't know."

I am a practitioner at heart. I will not use a system I do not trust, and I will not ask anyone else to either.

That's the reason this lab exists. If we're going to build toward autonomous defense, and we are, then the work of establishing what makes it trustworthy must be done, and it must be done publicly, where results can be widely scrutinized.

What we mean when we say AI

Over the past few years, "AI" has quietly come to mean "large language model" or "LLM plus harness." That collapse is convenient for marketing but bad for research and practitioners both.

When we use the term, we mean considerably more than LLMs or agents. We mean decision models. We mean neural networks and deep learning. We mean natural language processing as a discipline rather than a chat interface. Defenders face a variety of problems, and a great many of them are poorly served by handing them to a stochastic model and hoping. Some of the most useful things we can do for defenders are done with models that are predictive, not generative. We view AI and all its subtypes as tools to be used when appropriate, and we aim to use the data we gather to help practitioners decide which tool to use in which situation.

The five pillars

1. Offensive learning. Thinking like an attacker to discover what they learn, and how quickly, when new capabilities become available to them. What are the tells of an attack that uses AI, and how do both traditional and autonomous defense methods hold up against them?

2. Autonomous action. What the machine is allowed to do, when it is allowed to do it, and under whose authority. Scope, authority, rollback procedures, blast radius, and conditions for operation are all key factors here.

3. Decision models. Much security work needs to be deterministic, and it needs to be fast. Not everything should be inferred. Training small, narrowly scoped, deterministic models for repeated decisions comes with the added benefit of codifying existing but unwritten security procedures.

4. Trusted autonomy. The verification layer. This is the mechanism that lets someone who was not in the room, and who does not take the system's word for it, evaluate whether a defensive action was correct. This is the pillar the other four lean on.

5. Security context. A finding means nothing in isolation. The same vulnerability in two environments is two different problems, and we're past the point where triage can get better without robust organizational context.

I can't promise that these five pillars won't change over the lifetime of Kai Labs. Such is the nature of research. But each is a place where we think the evidence is currently too thin to justify the confidence being sold.

What you can hold us to

A charter is cheap, particularly in the age of LLMs. In my view, what makes a lab legitimate is whether its findings are specific enough to be proved wrong. We would rather publish a narrow finding that holds than a broad claim that collapses on first contact with somebody else's environment.

I'll make this commitment: we will publish what we find, including results that do not favor Kai's product, and we won't limit our research scope because we fear the answer. We will name our methods and our sample sizes. Where something is based on internal rather than public data, we will say so plainly, and we won't use that as a way to mask our own uncertainty. When somebody else's work changes our mind, we'll say so, and link back to the work in question.

You will be asked to trust autonomous defense sooner than you expect. We would rather that trust be built on data.

Kai Labs publishes at kai.security/labs. Everything we find goes there, including the results that do not favor us.

Kylie McClanahan is the Research Director at Kai, where she leads a team of researchers testing the limits of autonomous defense against agentic capabilities. She holds a PhD in computer science and has spent her career in vulnerability management, prioritization, and remediation, particularly in industrial control systems.