Is your human-in-the-loop just an Approve button?

I don’t think it’s enough. I could be wrong. Let’s check.

I can just tell you what I’d be looking for.

• Reviewers that approve almost everything.
• Metrics that don’t differentiate between AI that’s great and AI that’s not actually supervised.
• Approval queues that clear fastest on Fridays.
• Quality that “seems fine.”
• An embarrassing error that reached a customer, with little to no traceability.
• A major buyer’s question about how review works, answered by describing the button.
• AI features added to trusted products, with the new friction landing on whoever signs off.
• Service agreement updates mostly looking to pass the buck.
• Review growing as a cost of delivery, with no metrics to justify the increase.

If none of this sounds like you, I’m probably wrong about your product, and you can stop reading here.

Blame the system, not the people. They’re only human.

Automation has a long history of creating new problems the moment a person is involved. Aviation, medicine, industrial control, self-driving cars: decades of evidence say the human has to be designed in, not bolted on.

Attention fades when the machine is usually right or the task is repetitive. The fortieth decision of the day isn’t made like the first. And when the automation fails, the person has seconds to rebuild a picture they haven’t been watching. That’s the hardest moment in the job, and the least practiced.

Human-in-the-loop gets sold as a feature. It’s an engineering primitive. It can be measured and it can be designed. A policy does neither.

51 ways human review fails. Most never show up on a dashboard.

Each one has a name, a known cause and a known way to address it, worked out in every field where a person signs off on a machine. Inside most AI teams, it has none of those. It shows up as reviewer error, a missed edge case, or a customer complaint nobody can explain.

What you can’t name, you can’t measure. What you can’t measure, you can’t fix, and you can’t defend when a customer or a regulator asks how your review works.

A few you may not have a name for yet:

• Moral crumple zone. When something goes wrong, the blame lands on the reviewer who clicked Approve, though the design made the miss likely. Your customers’ staff may be sitting in one.
• Babysitting. You’ve given a person the one job people are worst at: watching a system that is almost always right.
• Anchoring. The reviewer sees the AI’s answer first. After that, they’re no longer forming their own.
• Premature closure. A plausible answer ends the search. Nobody looks for the better one.
• Forensic opacity. After an error, you can’t reconstruct what the reviewer saw or did. You can’t explain it, so you can’t fix it.
• Normalization of deviance. Small skipped checks cause no harm, so they become how the work is done.
• Reverse centaur. The AI was meant to assist the expert. Now the expert assists the AI.

Not sure how your product holds up against these? We should talk.

My offer: a tailored human review diagnostic.

One workflow, $2,500, delivered one week from kickoff.

• Why it fails. Your workflow and screens checked against 51 known ways human review breaks down. You get the ones that apply, named and ranked.
• What it costs. Where your reviewers’ time goes, in hours and dollars.
• What it catches. An estimated catch rate, with an honest range.
• What to change. Three recommended design fixes for the work that carries the most risk.
• A written report, yours to share. It’s evidence of how your review performs, not a certificate.
• A readout with your team, and a plan to measure again after you change things.

The cost and catch figures depend on your review logs and a few hours from one of your experts. If the logs aren’t there, you still get the findings and recommendations, plus a plan for what to record.

If the report gives you nothing you can act on, you pay nothing.

A pentest checks whether attackers can get in. A catch test checks whether errors can get out.

Your human review is a defence against your own AI’s mistakes. Few teams test it.

I test it the way airport security and clinical labs have for decades: by checking what got through.

The diagnostic does two things. It checks your workflow and screens for where review is likely to fail. Then it runs a catch test to see whether it does.

What a result looks like

What you should know before you call.

• I’m one person, and this is a new practice. You’d be among my first clients. That’s why the price is what it is.
• I have no client testimonials yet. I have years spent on this problem and a refund guarantee if the report gives you nothing actionable.
• I can’t prove your users and reviewers are paying attention. Nobody can. I can measure what they catch and what it costs. I can identify what sets them up to make mistakes.
• My numbers come with ranges, not decimals. A week of data gives an honest estimate.
• I might find your human review works well. If so, I’ll say so, and you’ll have evidence to show for it.

This isn’t for you if…

• You want a compliance certificate or a security audit. I’m not a lawyer or an assessor.
• You want a big firm’s name on the report.
• Nobody reviews your AI’s output. (Different problem. Possibly a bigger one.)
• You’d rather not know.

Why I am making this my problem

Yann GagnonPhysicist, PhD in medical biophysics

When deep learning arrived, the promise was that neural nets would replace radiologists. They haven’t. Hardly anyone asked why.

I founded ClearVoxel to build AI tools for radiologists, and found out. The startups were solving one kind of error: missing something on the image. Radiologists make at least a dozen others. They see a finding and dismiss it. They anchor on the clinical history. They stop looking after the first finding, or lean on a previous report.

AI fixed none of those sources of errors and created new ones: automation bias, attentional tunnelling, alarm fatigue.

Soon after, I saw the same pattern in self-driving cars, another industry making lofty AI promises. ClearVoxel didn’t survive 2020. The problems did, and I kept cataloguing them, from every field where a person signs off on a machine. The list now runs to 51.

Then LLMs arrived. Now every enterprise wants AI across its products and processes, with a human signing off on what it does. Regulators are writing that sign-off into law. Buyers are writing it into contracts. Teams are shipping it as a button. Getting it wrong will be expensive. Getting it right is a competitive advantage I can help you build.

Bring me one workflow.

20 minutes, no pitch deck, no obligation.

or email yann@crossfactors.co