AI in Claims Is Only Half the Conversation

AI can explain itself inside your firm. It goes strangely quiet the moment the claim crosses the street.

A point has been circulating the claims community lately—quietly at first, then with the peculiar persistence of something true but slightly inconvenient. It goes like this: when a claims professional can click through an AI recommendation and see exactly how the system arrived at its conclusion, they treat that recommendation differently.

Transparency changes behaviour. Accountability changes posture. The machine stops being a mysterious oracle and becomes more like a colleague who shows their work.

This is correct. It is also only half the conversation.

The half we’re having.

Right now, most of the industry’s attention is focused inward—inside the walls of a single organisation. The questions are sensible ones:

Can the handler see how the AI reasoned? Can the manager review the decision trail?

Can compliance reconstruct what happened after the fact without calling three people and opening twelve folders labelled “final_FINAL_v3”? These are exactly the right questions to ask.

Claims professionals already spend something like forty to sixty per cent of their time reading documents instead of exercising judgment, which is rather like hiring a seasoned negotiator and then assigning them to alphabetise filing cabinets.

Tools that surface risk signals, summarise files, and accelerate review are not replacing professionals. They are returning them to the work they were actually hired to do—negotiation, judgment, interpretation, and the delicate business of telling someone “we agree” or “we don’t” in a way that keeps the relationship intact.

This is real progress. But it turns out there is another half to the conversation, and it lives just beyond the firm boundary.

The half we’re not having.

Here is the question that keeps resurfacing once you start looking for it: When an AI-assisted decision leaves one organisation and binds another, what travels with it?

In the Lloyd’s market, the lead reviews the evidence and agrees the claim on behalf of the subscription. The followers accept. They don’t repeat the entire exercise. They rely on the lead’s judgment. This arrangement runs on institutional trust, and remarkably, it works.

But if the lead used AI in that decision, the followers don’t know. They don’t know the system flagged something. They don’t know whether the handler followed the recommendation. They don’t know whether the handler ignored it entirely while drinking tea and instead trusted instinct.

Inside the lead organisation, the decision is auditable.

Outside the organisation, the AI disappears like a stage magician stepping behind the curtain.

And Lloyd’s is hardly unique. The same thing happens with co-insurers on shared risks. With reinsurers relying on cedents. With carriers overseeing MGAs under delegated authority. The decision crosses the boundary. The reasoning stays home.

The boxcar problem

A senior architect from ACORD gave me the best description of this situation I’ve heard so far.

He called it the boxcar problem.

The industry has spent decades building excellent tracks. Standards move structured data between systems with impressive reliability. Messages arrive where they are supposed to go.

Fields line up. Numbers match.

But the context behind decisions—who reviewed what, under which standards, using which tools—has no carriage attached to it. There are roughly three thousand carriers in the world. Each does things slightly differently.

Everyone agrees the cargo is important. No one has installed the lock on the boxcar.

So the decision arrives. The reasoning does not.

Why this matters specifically for AI

The transparency argument about AI is exactly right. When professionals can see how a recommendation was formed, they engage with it differently. They challenge it. They refine it.

They occasionally ignore it with confidence rather than suspicion.

But that transparency stops at the firm boundary. Inside the organisation: the handler sees the recommendation, the manager reviews the trail, and complies with the process.

Outside the organisation: followers see a number, reinsurers see a status, oversight carriers see an outcome. No one sees the reasoning. No one knows whether AI participated. No one knows whether it was followed.

So when we talk about “trustworthy AI in claims,” the obvious question quietly appears: trustworthy for whom? The handler? Or the five downstream parties already bound by the decision?

What I learned from supply chains

Before I came to insurance, I spent years researching multi-party coordination in supply chains. I built a platform that let parties in the chain see what happened before them. Nobody used it. Not because the technology was wrong, but because the design was. It asked busy people to do something extra. The lesson: if adoption depends on users choosing to engage, they won't.

Infrastructure that creates a record quietly, as a byproduct of existing workflow, is a fundamentally different proposition.

The claims professional never sees it. Quietly. Automatically. Without requiring anyone to volunteer for additional responsibility at 4:47 on a Friday afternoon. 

But the day a regulator asks, or a reinsurer queries, or someone needs to reconstruct what happened eighteen months ago, the record is already there.

The AI governance question across firm boundaries won't be solved by asking downstream parties to audit upstream decisions. It will be solved by infrastructure that captures the decision context at the point it's made and makes it portable without adding a single step to anyone's day.

The question nobody is asking yet

The insurance industry is investing heavily in making AI trustworthy inside organisations. That instinct is correct. But trust inside a firm and trust between firms are not the same engineering problem.

We solved the data exchange. We are solving internal AI auditability. The unanswered question—the one still pacing around outside the meeting room—is this:

How does trust in AI-assisted decisions travel between firms?

I don’t have the full answer yet. I’m about a year into researching the problem through the Verida Charter Foundation, and I am still pressure-testing the thesis with practitioners who are generous enough to tell me when I’m wrong.

But the pattern keeps repeating. The industry built excellent tracks. The cargo is getting heavier.

The lock for the boxcar still doesn’t exist.