20 - Chapter 20: The Illusion of Validity

Chapter 20: The Illusion of Validity

The Illusion of Validity

·          e.g. watch soldiers at task, try predict candidacy as officers, observ’ns seem compelling (despite little time, artif’l situ’n) à very confident judg’ts —but later performance as officers show predic’ns almost worthless — but still confident making such judg’ts — substit’n + repr’ness heuristic, no regress., WYSIATI
·         confidence is a feeling, reflects coherence of info + cogn. ease of processing it — confidence not reflect soundness of judg’t
·         illusion of validity, a cognitive illusion

The Illusion of Stock-Picking Skill

·          stock market built largely on an illusion of skill — in theory, both buyers & sellers hv same info, so current stock price incorporates all info about co.’s value as well as predic’ns of its future —  i.e. stock prices are perfect, so why one person buy, another sell?
·         studies show that indiv’l investors on avg buy & sell badly, sell growing stocks (wh. contin. to grow), hold losers (wh. contin. to lose), too much attention to news stories — more trading = more loss
·         pros take advantage of amateurs’ mistakes — but even pros (incl. fund mgrs.) do not succeed consistently, 2/3 of mutual funds underperform the market every year
·         success ß luck, not skill  

 What Supports the Illusions of Skill and Validity?

·         they are stubborn illusions — assess future perform. of soldiers, invest. in stocks, illusion of skill endures — pro. investors do great deal of research & analysis à equate that w. skill in outcomes (ignore fact that stock market already incorporates the research they do in current stock price)
·         prof’l culture supports the illusion of skill  

The Illusions of Pundits

·         in hindsight we constr. a narrative that makes sense à think event was predictable à believe future much more predictable than it is
·          no “march of history,” events are random — e.g. 50% chance that Hitler embryo wd have been female
·         Philip Tetlock, Expert Political Judgment: How Good Is It? How Can We Know? (2005), tested  pol’l & econ. pundits, chose among 3 possible outcomes, predict probabilities, result was less than random — more knowledge à slightly better predict’ns than others, but very knowl. are overconfident, excuses, hate to admit errors

 It is Not the Experts’ Fault—The World is Difficult

·          conclusions:

o   (1) errors of predic’n ß unpredict’l  world, errors inevitable
o   (2) feeling of confidence does not indicate accuracy
·         short-term predic’ns are poss. w. some accuracy, not long-term   

Speaking of Illusory Skill

·         “He knows that the record indicates that the development of this illness is mostly unpredictable. How can he be so confident in this case? Sounds like an illusion of validity.”
·         “She has a coherent story that explains all she knows, and the coherence makes her feel good.”
·         “What makes him believe that he is smarter than the market? Is this an illusion of skill?”
·         “She is a hedgehog. She has a theory that explains everything, and it gives her the illusion that she understands the world.”
·         “The question is not whether these experts are well trained. It is whether their world is predictable.”