Over on OUPBlog, Martin Smith wrote a blog post, related to his book “Between Probability and Certainty: What Justifies Belief” (that I haven’t read yet). He presents an example in which, he claims, it is rational to believe the unlikely. Please read his blog post first, then return here to read my reaction below. :-)
Dear Martin Smith,
I enjoyed thinking about this example, but I do not agree with the conclusion. I will share my reasoning here, so maybe you can convince me otherwise.
Laplace wrote in his Essai on probability theory:
“Plus un fait est extraordinaire, plus il a besoin d’être appuyé de fortes preuves. Car, ceux qui l’attestent, pouvant ou tromper, ou avoir été trompés, ces deux causes sont d’autant plus probables que la réalité du fait l’est moins en elle-même.”
As I am sure you know, he wrote this as an informal explanation of what we now call Bayes’ theorem. He follows the remark by an example of an urn and a witness, who may be mistaken (analogous to your example). Laplace generalizes the conclusion of the example, stating that the probability of a mistaken witness reports becomes higher as the event becomes (a priori) less likely.
In your example:
- “only 1% of the population are glasses-wearers” and
- “if the person wasn’t wearing glasses, there is a [2%] chance that the shopkeeper would have mistakenly said s/he was”
You also state: “It’s still true that there would have to be some explanation if the shopkeeper’s testimony were false and, having no inkling of any such explanation, I owe it to him to accept it.”
However, in the example, it is more exceptional that a person wears glasses than that a shopkeeper gives a mistaken report, so wouldn’t the alleged observation that the person was wearing glasses be more in need of an explanation? It strikes me as odd that only possible explanations are given for the less unlikely event: “because of a hallucination, false memory etc.” I guess this is the point we disagree on, since you write there is nothing hard to believe about the fact that a glass-wearing person could have been the kind stranger. I agree that in the scenario there is no reason to disbelieve a glass-wearing person could do a random act of kindness, but it is far less likely for a glass-wearing person to be around in the first place. So, there may still be a base rate fallacy in the background. And it is only the part that this person was around that requires extra evidence or explanation (assuming the base rate probability for doing random acts of kindness is constant over the population). Such an explanation could be as simple as s/he lives here, or was visiting/shopping/…
In the example, I wouldn’t (yet) believe that the person wore glasses, although I fully agree that it would be impolite to call the shopkeeper a liar (and actually, lying isn’t among the mentioned hypotheses to explain a faulty report). It would simply have become much more likely than before talking to him. Now the option that the person was a glass-wearer has become worth considering with more attention, so the report is definitely relevant for informing us about what may have happened. But why should we not follow Laplace and look for further corroborating evidence rather than believing a less likely conclusion over a more likely one?
The fact that we should be grateful to the shopkeeper for speaking up and the fact that it would be impolite to openly doubt his statement seem to be a different matter than what is rational to believe. Since the shopkeeper’s report makes you raises the probability you assign to the glass-wearer hypothesis from 1% to 33.5%, it’s not a matter of “refus[ing] to believe what someone tells me” or dismissing it! It’s just a matter of not jumping to conclusions based on a single source.
Maybe we should also distinguish between two cases: whether or not the shopkeeper knows that there are few glass-wearers around. He may not know the statistics, but may know implicitly if he worked in the neighbourhood for long enough. If he has just moved there, he may not have noticed yet. If he knows that glass-wearers are rare, he may have been surprised and looked up an extra time and/or be extra careful in reporting this. This may decrease the probability of misreporting this, but then the assumed base rates don’t apply to him and the relevant computation would require different numbers.
A further complication is that the base rate for glass-wearers is prefaced by “I then remember reading that”, which introduces additional probabilities regarding the accuracy of the source and your own memory.