Research & analysis: 20 ways AI gets it wrong
Shallow research presented as deep, cherry-picked evidence and missed changes. Each failure mode below is phrased as the question people actually ask, with what it looks like in real work — and the layer of the Trust Stack that catches it.
Does AI present shallow research as if it were deep?
A quick skim is dressed up with the confidence of an exhaustive investigation.
Why does AI stop researching before it's done?
A conclusion is drawn after the first result, missing better ones further down.
Can AI search the wrong terms and miss the answer?
The real answer is missed because the search used the wrong vocabulary.
Does AI miss sources in other languages?
The most authoritative source is overlooked because it isn't in English.
Why does AI miss official bulletins and amendments?
A recent amendment in a government bulletin is never picked up.
Can AI miss a recent change in policy?
Advice follows a policy that was changed weeks ago.
Does AI miss how an industry actually works in practice?
A textbook answer ignores how the field really operates day to day.
Can AI confuse being eligible with being approved?
A note treats meeting the criteria as if approval were already granted.
Does AI confuse what's possible with what's likely?
A remote possibility is described as a probable outcome.
Why does AI confuse correlation with causation?
Two trends that move together are presented as one causing the other.
Can AI draw conclusions from incomplete data?
A firm recommendation rests on a dataset with obvious gaps.
Does AI fail to stress-test its own conclusions?
A conclusion is delivered without any check of whether it survives scrutiny.
Why does AI cherry-pick evidence that fits its answer?
Only the supporting data is shown while contradicting data is left out.
Does AI ignore the edge cases that break a plan?
A solution works for the typical case but fails the unusual ones it never considered.
Can AI ignore how hard something is to implement?
A recommendation reads simply but would be very hard to actually carry out.
Does AI fail to say what it still doesn't know?
An answer feels complete but never names the open questions that remain.
Why doesn't AI rank how strong its evidence is?
Strong and weak evidence are mixed together with no sense of which is which.
Does AI hide the assumptions behind its answer?
A recommendation depends on assumptions that are never stated.
Can AI give a tidy answer to a messy problem?
A complicated situation is reduced to a clean answer that hides the real complexity.
Does AI recommend things without enough context?
Advice is given before the specifics that would change it are understood.
More from the library
One model can’t reliably catch its own mistakes. A council of independent minds can.
Run your work through the councilAll 250 failure modes · See also: the Trust Stack