The illusion of knowledge: why more research can make you worse

More information feels like more control. In one of the most-cited studies in the science of judgment, it bought exactly one thing, confidence, and none of the thing you actually wanted: accuracy.

In 1973 the psychologist Paul Slovic ran a now-famous experiment with eight professional horse-race handicappers, people who bet for a living. He had them rank the horses in forty races, first using five pieces of information per horse, then ten, then twenty, then forty. At each stage he recorded two things: how accurate they were, and how confident they felt.

Confidence and accuracy come apart

The dangerous finding isn’t that extra information failed to help. It’s that it made the experts feel like it helped. With five variables their confidence roughly matched their accuracy, well calibrated. By forty, they were badly overconfident: certainty had kept rising while correctness had flatlined. Later researchers found the same pattern again and again, adding information reliably inflates confidence far more than it improves judgment.

Why this is expensive for investors

Here is the chain that makes it costly. Overconfidence is what drives people to trade more, and trading more is what drives them to underperform (the subject of a separate essay). Every additional chart, forum thread, earnings hot-take, and podcast feelslike accumulating edge. Mostly it’s accumulating conviction with no edge attached, and conviction is precisely the thing that gets you to act.

Name the few inputs that matter

The fix isn’t to know less. It’s to decide, in advance, which small handful of inputs a given decision actually rests on, and treat the rest as what it usually is: noise dressed up as diligence. A rule that states its inputs plainly (“rebalance if any position exceeds 25%”) is really a sentence that says: these are the variables this decision depends on; the other forty are for confidence, not correctness.

That’s part of what writing a rule does for you in Cetagon, it forces the decision down to the inputs that matter, and keeps you from drowning them in the other thirty-nine. Turn a decision into a rule →