"Noise" in organizational decision-making
Daniel Kahneman's work on "fast" and "slow" decision-making is famous. With Olivier Sibony and Cass Sunstein, he explores inconsistencies in professional judgement and how it affects organisations.
Kahneman, Sibony and Sunstein identify what they call noise - persistent inconsistencies in professional decision-making. Their book is Noise: a flaw in human judgment and there is an interview in New Scientist: The biggest flaw in human decision-making and how to fix it.
Noise is variability of conclusions among highly skilled professionals like doctors and judges when presented with the same information.
“[Organisations] are designed to sweep the problem under the rug and to create the illusion of consensus. They are not looking for the correct answer.”
While they acknowledge it’s expected that individual judgements will differ, they demonstrate that they vary far more than expected. In a standardised test of insurance underwriters, they expected their judgements would be around 10 percent different on average but they varied by 55 percent. Similar results were found for sentencing decisions by judges (50%).
Noise versus bias
Bias can apply to a single case, and it may be possible to identify sources of bias. Noise is the statistical variability among cases, where the judgements should be very similar, it turns out they aren't. This has implications for the quality of outcomes and fairness of decision-making across an organisation.
Hiding the noise
You have to ask: why don’t organisations realise this problem? Why doesn’t the insurance company realise it? Why doesn’t the judicial system become aware of so much variability? Why don’t hospitals become aware of the fact that doctors have, quite often, very different diagnoses of the same patient? You would think it’s in their interest.
The most striking passage in Kahneman and Sibony's New Scientist interview is this (my emphasis):
We think part of the answer is that organisations are designed to suppress evidence of noise. They’re designed to sweep the problem under the rug and to create the illusion of consensus. They are not looking for the correct answer. One way to do that is never ask people for their opinions separately. You bring them into a meeting and you ask them to discuss it. Which, of course, gives a strong incentive to the second speaker to agree with the first one, and the third person to agree with the first two, and so on.
Some thoughts on the implications
I am not sure this is true of every organisation but I can relate their conclusions to my experiences.
The Delphi method predates this work and tries to provide a structured way to address the issues Kahneman, Sibony and Sunstein raise.
Companies should carefully consider their diversity and inclusion efforts to help maximise their chances of making robust and fair decisions and ultimately benefiting the company as a whole.