Would You Trust a Mirror for a Second Opinion?


AI wrote the recommendation, another AI checked it, and a third AI approved it, leaving us with three green checkmarks and the comforting feeling that somebody has been responsible. Ahh...due diligence! We automated the entire echo chamber, and it even provided an explanation of why we should trust it. 🤣

What makes me squirm is how easily that sequence can feel like evidence accumulating. Each approval adds weight, each explanation adds polish, and somewhere along the way we stop asking whether anything new has actually been established. (I can feel myself wanting to trust the checkmarks, which is precisely why this bothers me.)

Imagine five executives approving a proposal after reading the same flawed report. Their agreement might tell us something about how persuasive the report is, but the original error survives every signature. Now imagine those executives can approve ten thousand proposals before lunch, write beautifully reasoned defenses of each one, and trigger the decisions themselves.

What did “verified” actually mean?

Before celebrating the architecture, I would want answers to three questions:

  • Checked against WHAT? Identify the source, measurement, rule, or test that could establish whether the claim holds.

  • What could make the reviewer reject it? Feed the process known mistakes, especially errors the generating model tends to make. A reviewer that applauds good answers but also approves convincing nonsense has a rather important performance problem.

  • Did we add independent evidence, or another interpretation of the same material? Different providers can help, but different logos do not establish independence. Trace where the evidence came from and whether every reviewer inherited the same assumption. (Hiring three people to read the same incorrect spreadsheet does not repair the spreadsheet.)

The research behind this concern is uncomfortable. In a 2026 preprint, Guneet Kohli tested nine frontier models from seven model families across three natural-language-inference datasets, using reference labels derived from human annotations. Based on correlations in their errors, the panel supplied approximately 2.2–2.5 independent votes’ worth of information. That is an estimate of informational independence, rather than a claim that only two models did any work. The models often failed on the same items. On one dataset, the panel achieved 72.0% accuracy against the strongest individual judge’s 71.8%, a negligible improvement; it performed worse than the best individual on the other two. These findings concern particular benchmarks and models, not every AI review system, but they challenge the assumption that adding reviewers automatically adds meaningful scrutiny.

Nine judges, roughly two independent votes...that is one expensive meeting for such a small increase in perspective!!! What troubles me more is that an organization might count all nine approvals without ever measuring what the additional eight contributed.

AI can manufacture the reassurance, too

This is where I think AI changes the problem beyond simply automating an existing process. A conventional rule-based check might compare an invoice total with a purchase order. Its rules can be wrong and its data can be garbage, but we can inspect the comparison it was instructed to perform. An LLM reviewer can be asked to interpret whether a recommendation is reasonable, decide which considerations matter, and generate an explanation for its verdict. That flexibility is valuable, but it also means the system can produce the answer AND the reassuring argument for accepting it. If the writer and reviewer share a blind spot, the explanation can make the mistake feel better examined than it actually was.

A second study, published at NeurIPS in 2023, helps explain my unease. Miles Turpin and colleagues tested GPT-3.5 and Claude 1.0 with prompts containing biasing cues. The models could produce explanations supporting incorrect answers without mentioning the cues that influenced them. The lesson I take from those experiments is that a plausible explanation is not necessarily a faithful account of how an answer was reached. And that feels consequential...because explanations are one of the main ways we decide whether someone deserves our trust. We ask people to show their reasoning, then instinctively carry that habit into our interactions with AI. (I WANT explanations. I also want to remember that an explanation is something we may need to evaluate.)

The philosophical question underneath all this is what we believe verification owes us. Agreement describes a relationship among judgments; truth concerns whether those judgments hold up against what they describe. A room full of agreement can still have no useful contact with the thing being discussed. The uncomfortable possibility is that we become very good at recognizing the appearance of certainty while losing the habit of questioning its basis.

In manufacturing, ten systems agreeing that an inspection occurred cannot perform the missing inspection. If they all rely on one inaccurate AI-generated summary, their agreement adds no new observation of the machine. Once agents can act on that agreement, a questionable claim can become an operational decision before anyone notices the gap.

I remain excited about AI helping us verify work, find contradictions, run tests, and examine evidence at a scale people cannot manage alone. The opportunity is enormous, provided we evaluate reviewers by the errors they actually catch and preserve a path for contradictory evidence to stop the process. Human involvement deserves the same scrutiny; a person clicking “approve” can be another mirror. For me, “verified” should carry an obligation: we can explain what was checked, against what, and what remains uncertain. Otherwise, we risk building systems that become more confident with every conversation while learning nothing new about the world.

If every AI is checking another AI’s answer, when does reality get a vote? Preferably before the agents have already acted on the result. 😇


References:

  • Kohli, G. (2026). Nine judges, two effective votes: Correlated errors undermine LLM evaluation panels [Preprint]. arXiv. https://doi.org/10.48550/arXiv.2605.29800

    Turpin, M., Michael, J., Perez, E., & Bowman, S. R. (2023). Language models don’t always say what they think: Unfaithful explanations in chain-of-thought prompting. Advances in Neural Information Processing Systems, 36, 74952–74965. https://doi.org/10.52202/075280-3275

Next
Next

What Makes a Knowledge Graph Different from the Systems We Already Have?