Less than amazing: human error under AI-assisted scrutiny

I was going to illustrate this point with a structural engineering example but I would test everyone's patience, so let's turn to something more relatable.

Imagine a publisher in 1995 producing a 1,000-page technical textbook. Two experienced proofreaders carefully examine every page. The book is published, but some errors remain, including in mathematical formulas.

Over the years, readers occasionally flag a handful of mistakes, which are corrected in subsequent editions.

Fast forward to 2026. The publisher gets two independent AI agents to proofread the original book. Within minutes, they identify 50 previously undetected errors, including some pretty serious and potentially consequential ones.

All of those errors were theoretically detectable in 1995. So why weren't they detected?

Because human checking has inherent limitations: attention, fatigue, complexity, time constraints and the sheer volume of information to be examined mean that even the most diligent reviewer will miss some mistakes. And we have learned to live with imperfect books and tolerate them, as most of the time we barely even notice.

Does the discovery of these 50 additional errors mean that the original proofreaders were incompetent, or that the publishers' QA process in 1995 was inadequate? I don't think it does.

Curiously, the same questions seem harder to answer if we turn to the way AI may be used to review historical (structural) designs produced without it, detect errors and investigate construction disputes. Why?

For one, because serious errors can have grave consequences, and once a serious error is detected, the means of detection seem to matter less. Of course, it is not fair to exercise judgement mediated by the manifested consequences on one's competence, but that's an old question that we do not need to get into here. What changed is this: with powerful mechanical means of scrutiny at our disposal, witholding restrospective judgement becomes much – much! – harder than it has ever been.

Until now, the main obstacle (besides the inherent hindsight bias) has been the focal depth of our analysis: when called upon to review in depth a fraction of the information originally produced, we concentrate analytical effort on one element of the original engineer's much larger undertaking, making it difficult to maintain a balanced view on what quality means. But at least humans were judging the work of other humans, and there was something reassuring about that; we operate under the same human limitations forcing our judgement into perspective.

How do we navigate a landscape where the work of humans is scrutinised by AI agents, and the findings can carry serious legal consequences?