Let’s start with the scientific context. Replication seems like a scientific virtue. The more instances of corroboration we have the stronger our evidential set seems to be. But replication is not always virtuous. Here’s a simple example.
For a few, very exciting, months it seemed as though physicists had detected particles that travelled faster than the speed of light. A group of physicists generated muon neutrinos at the Super Proton Synchrotron (SPS) particle accelerator at the CERN LHC complex in Geneva (Amelino-Camelia 2012). They accelerated these neutrinos down a 1 km beam line toward the Gran Sasso National Laboratory in Italy. Incredibly, neutrinos travelling from CERN to Gran Sasso were measured to make the trip 60.7 nanoseconds faster than light speed. The measurement procedure was very carefully planned. For example, time was measured with GPS timing signals and a cesium atomic clock. The GPS also allowed the physicists to track any small movements in the Earth itself. Even factors such as day vs. night and seasonal trends were taken into account. One of the most important and interesting facts about the methodology is that the experiment was repeated 15,000 times! Additionally, later experiments, checking for systematic error by fiddling with different proton pulse profiles, seemed to replicate the results.
It seems as though each trial (and each replication) serves as a “check” on the other trials. However, this fails to be case of good methodological checking. The reason is the repeated/replicated measurement procedures are not independent. It was discovered (Cartlidge 2012) that the 60 nanoseconds discrepancy came from a poor connection between a fiber optic cable that connects the GPS receiver (used to correct the timing of the neutrinos' flight) with an electronic card in a computer. The connection was tightened and the faster-than-light neutrinos were no more. Each of the 15,000 trials could not serve as a “check” for other trials because they all suffer from the same systematic mistake. To properly check results, we need independence of modes.
The independence of modes is something that we can experiment on in a very simple way. Have a large group of people guess your weight without debating with each other about any of the factors. The likely result is that when you add up and average everyone’s guess, you will get a number very close to your actual weight. The reason why this works is because some people guess low, others guess high, and the error cancels out. But each person’s error is independent of another person’s error. As long as this is the case, then multiple heads are better than one.
But what if you’re not dealing with multiple individuals or multiple sources of information? How does independence of modes apply to you? The title of the post is, after all, about you checking yourself. The next example will give us a simple view of modes and independence.
MacArthur Wheeler robs a bank in broad daylight (Kruger and Dunning 2009). He does not wear a mask and is captured within an hour. When captured, he is in complete shock, “But I wore the juice.” Wheeler believed that putting lemon juice on his face would make him invisible to cameras. He arrived at this conclusion by, what seemed to be, good checking methods. He put lemon juice on his face and took a series of selfie-polaroids. In this case, each photo seems to be an independent mode of measurement. In the results of the polaroids, Wheeler was nowhere to be found. How could all of the photos converge on this result? Simple: What compromises the independence of each measurement process is Wheeler’s systematic stupidity. Wheeler is the engineer behind each measurement process and the interpretation of each result. As such, he is part of the design, implementation, and interpretation of each measurement process. So whatever mistakes he makes, he continues to make, unless an independent engineer steps in and corrects them. This is what is referred to as the Dunning-Kruger Effect: general incompetence produces lack of awareness of incompetence. Think for a moment, though: consistent with the Dunning-Kruger effect, checking yourself is something that everyone can reflect on, and nothing that anyone can truly do for themselves.
We've come to the conclusion that checking requires the independence of modes. We have illustrated what kinds of independence are good for checking. The next step is defining independence of modes. Not only is it important to specify conditions for independence, but it is also important to characterize what kind of independence is sufficient (e.g. partial vs. full independence).
Amelino-Camelia, G., (2011). OPERA data. Phenomenology of Philosophy of Science. arXiv: 1206.3554v1[physics.hist-ph]
Cartlidge, Edwin, (2012). BREAKING NEWS: Error Undoes Faster-Than-Light NeutrinoResults.
J. Kruger, D. Dunning (2009). Unskilled and Unaware of It: How Difficulties in Recognizing One's Own Incompetence Lead to Inflated Self-Assessments. Psychology, 1 (2009), pp. 30–46
Department of Philosophy