Noise: A Flaw in Human Judgment

Part 3 evaluates “the quality of predictions,” assessing how accurately predictions “co-vary” with outcomes (130). For example, in the practice of hiring for, employers can compare the potential observed in employees by HR departments and their performance a few years later. Accurate predictions will show the estimate of high potential resulting in great achievement. This section also studies the impact of noise on predicting events and evaluating them when they occur.

In employment, performance prediction is an essential aspect of hiring. Candidates are often given a numerical rating on attributes such as leadership or interpretative skills, and theoretically the highest scoring candidate ought to be the best for the job. However, a doctoral study of performance prediction based on numeral ratings showed that the percent correlation of a prediction of good performance with actual good performance was 55 percent and therefore “barely better than chance” (134).

In 1954, psychology professor Paul Meehl reviewed 20 studies in which clinical judgment, the standard form of human judgment, was compared with mechanical predictions of outcomes like academic success and psychiatric prognoses. He concluded that mechanical rules were more reliable than human judgment because humans overvalue certain qualities, whereas mechanical prediction distributes equal weight to relevant qualities.