By Arrash Nekonam, Chief Technology Officer, COMET

Every serious incident investigation ends with the same implicit promise. We looked at what happened, we understood why, and we have taken actions to make sure it does not happen again. That promise is the entire point, not the report, not the actions derived from the process, but the understanding held by real people and informing real decisions. That understanding is what turns investigation into prevention.

So if AI is doing the thinking, is that promise still being kept?

We are about to find out the hard way

AI is being deployed across industries and workflows at a pace that has, understandably, outrun some of the harder questions. Procurement decisions are being made on the basis of speed, cost and feature lists, all legitimate considerations, but very few of them address what actually determines the value of investigation over time, which is whether the people doing it are learning from it.

A misconception worth naming

There is an assumption embedded in a lot of AI product development right now that deserves to be examined carefully: that capability equals understanding. That because a system can produce a credible causal analysis, it has understood the incident in any meaningful sense.

Having spent a long time working in software architecture and AI systems, I want to be clear about what I believe is actually happening. The system has processed patterns, matched inputs against prior data, and generated outputs that, in many cases, will be accurate and useful. But understanding, in the way that supports prevention, requires context that lives outside the data, judgement that comes from years of domain experience, and the kind of pattern recognition that an experienced investigator carries into a room before the analysis has even begun, the instinct that tells them something systemic is at play long before the data confirms it.

Root cause analysis was designed to produce that understanding, and the difference is really important when you think about what happens next. An investigator who has worked through evidence, tested hypotheses, challenged assumptions and arrived at a causal conclusion has learned something durable. They understand how that failure happened and why, they will recognise the conditions if they start to appear again, and that knowledge lives in them, not in the report. Signing off on someone else's reasoning is not the same as doing the reasoning. The report will not show that distinction. The next incident might.

Scale that dynamic across a team, across a year, across a generation of practitioners who have come up in an environment where AI handles the analysis, and the problem becomes structural. The reports look the same, the action rates may even improve, but the question of who actually understands failure in that organisation, who can look at a near miss and recognise what it is the precursor to, has a quietly different answer. And it will keep having a different answer until something serious happens again.

What AI can do is free up the people who have that expertise to use it more and use it better. That alone justifies the investment. But it depends on a clear-eyed understanding of what AI is actually doing, and what it is not.

Automation bias is not a flaw, it is simply how human attention works.

Every sufficiently capable automation system produces the same effect over time. The more reliably it performs, the less thoroughly it gets checked. Scrutiny that once felt necessary starts to feel excessive, and effort calibrates accordingly. In most domains that is a rational response to reliable automation.

In incident investigation it becomes a slow-building problem. The cases where AI is most likely to be subtly wrong, the complex, ambiguous, systemically significant events, are precisely the cases where human expertise matters most and where the cost of disengagement is highest. A workflow designed so that humans sit at the end of an AI process and call it governance does not address that dynamic. It creates ideal conditions for it.

Before asking which AI to buy, organisations should ask whether they are ready for it

AI adoption in investigation is usually framed as a technology selection question, but the more important question comes before that. Organisations with weak investigation cultures, inconsistent methodology or limited investigator capability will not find that AI strengthens those foundations. They are more likely to find that it papers over them, producing faster and more presentable outputs while obscuring the fact that the underlying process was never generating real learning in the first place.

The organisations positioned to benefit are those where the fundamentals are already in place, where investigators are trained and supported, where methodology is consistent, and where there is real appetite to understand failure rather than simply record it. For those organisations, well-designed AI can be transformative.

For those that are not yet there, investment in investigation capability is likely to deliver more than investment in AI, and the sequencing of those decisions matters more than most procurement processes acknowledge.

The question worth asking of every vendor

There is a version of AI-assisted investigation built on the premise that expertise is a bottleneck, that the goal is to reduce dependence on skilled investigators and make the process more scalable. In high-risk industries, that logic produces a productivity argument that carries a safety cost nobody has properly priced in.

The right design question is how AI can reduce the burden on investigators without reducing their engagement, and those are different problems that produce different tools.

This difference is beginning to attract serious analytical attention. Verdantix recently examined how AI vendors approach human-in-the-loop design across enterprise safety software, drawing a meaningful line between passive oversight, where humans sit at the end of an AI process, and active governance, where investigators are structurally required to engage at every decision point. COMET was one of a small group of vendors Verdantix cited for active human-in-the-loop approaches, alongside Moxo, Zapier, Cobbai, Duvo and IFS.

The full recognition reads:

'COMET embeds human validation as a structural requirement throughout investigation workflows. AI agents accelerate data retrieval, evidence correlation and orchestration but cannot advance a case without explicit user confirmation at each decision point, ensuring investigators retain authority over findings and causal logic.'

You can read the full Verdantix piece here.

That recognition reflects decisions we made deliberately. When we were building the COMET AI Assistant, we had many options about how far to take the AI capability. We made a conscious choice not to let AI perform root cause analysis or human factors analysis. AI can prepare, organise and suggest, but the causal reasoning sits with the investigator. Not because we could not, but because we felt we should not. The methodology in those areas exists precisely to keep the investigator doing the reasoning, and introducing AI at that point would have undermined the thing we were trying to protect.

We are also candid about a risk that any organisation adopting AI in investigation should take seriously: over-reliance. Investigation skill fade is a genuine concern, and it is one we think about in how we design and how we advise our customers. The skill of investigating cannot be replaced by AI, and that is not what we are trying to do.

The question that deserves a boardroom answer

In five to ten years, when the most experienced investigators in your organisation have retired or moved on, will the people who replaced them be able to investigate, not to review an AI output, but to reason through evidence independently, to challenge a causal narrative, to recognise when something does not add up? If the answer to that question depends entirely on what your AI does or does not do for them, then the investment decision being made now is also a workforce decision, and it deserves to be treated as one.

The organisations that get the most from AI in investigation will not be the ones that automated the most, but the ones that were clearest about what investigation is for and most deliberate about protecting it.

If this resonated with you

We explored these questions in depth in the latest episode of The Risk Factor, our podcast series on the conversations the industry needs to have. Episode 3, One technology, two truths: the AI conversation we need to have, is available to listen to now here.

If you would like to see how we have put these principles into practice, you can learn more about the COMET AI Assistant here.

About the author

Arrash Nekonam is the Chief Technology Officer at COMET, where he leads product strategy and technical architecture for the company. He has spent over 20 years building software for safety-critical industries, including offshore energy, maritime and manufacturing. A passionate technologist and founder, he bridges commercial vision with deep engineering expertise. Under his leadership, COMET delivers scalable, secure, and intelligent solutions that empower global clients to improve safety, compliance , and operational performance.