Wednesday, April 9, 2014

Signatures from Experts don’t mean anything

One of the worst ways of establishing an expert consensus, or more often a non-consensus of experts, is with signatures. Many find a long list of names persuasive for establishing that there is at least conflict within the field. 300 signatures from biologists stating that they don’t believe in evolution, or economists stating that they don’t believe in free trade, or climatologists stating that they don’t believe in global warming, is not good evidence that there is conflict in the field. Why? Because there are millions or sometimes hundreds of thousands of experts, and those who put their name on it are not a representative sample of the whole field. The denominator is too high (and of course not mentioned) for some large list of names to mean anything.

How many names one gets on a piece of paper ends up being a strong indicator of how much work one is willing to do, and a very weak indicator of what the experts believe.

Oftentimes what qualifies as an “expert” is broadened in order to get more names anyway.