18 Aug 2005 - 00:45

statistics and the law

Here is the heart of John Kay's column about the counterintuitive nature of Bayesian statistics:

Last month, the General Medical Council struck off Professor Sir Roy Meadow, the paediatrician, from the medical register. He had given misleading evidence in the criminal prosecution of Sally Clarke, whose two infants died in their cots. When Mrs Clarke was charged with their murder, Sir Roy told the jury that the chances of two successive cot deaths in the one family was “one in 73m”.

But although the disciplinary committee heard evidence from distinguished statisticians, it does not appear that they understood the application of probability theory to such cases any better than Sir Roy. The committee found that he had underestimated the incidence of cot deaths, and that he had not taken account of genetic and environmental factors that mean a household that experiences one cot death is more likely than average to suffer another. But even if you recognise these effects, his key conclusion remains valid. It is unlikely that such an accident would have happened at all. It is very unlikely indeed that such an accident could have happened twice in the same family.

Of course it is unlikely. The events that give rise to criminal cases are always unlikely, otherwise the courts would be unable to deal with the backlog. If Osama bin Laden is ever brought to justice, the question will not be “is it likely that two aircraft hit the World Trade Center on September 11?” – to which the answer is no – but “given that two aircraft did hit the World Trade Center on September 11, is it likely that bin Laden was responsible?” Confusion of these two separate issues has become known as “the prosecutor’s fallacy”.

A cot death in a family increases the probability that there will be another, but a murder in a family may well increase the probability of another murder by even more: wicked parents may continue to be wicked. Sir Roy might have been right to conclude that two cot deaths were more suspicious than one. But the Court of Appeal, releasing Mrs Clarke, was certainly right to have concluded that this statistical evidence could never, on its own, establish guilt beyond reasonable doubt.

You should not trust doctors, or lawyers, with probabilities; and be very hesitant about trusting yourself. Adversarial legal proceedings are a bad forum for unravelling technical issues. And we cannot expunge collective responsibility for mistakes by excoriating selected individuals.

The business and financial system, more than Bernie Ebbers and Henry Blodget, was to blame for the dotcom boom and bust. Failures in legal processes, rather than over-confident professors, led to the unjust conviction of women such as Sally Clarke.

I'm going to add this story to my collection of Cautionary Tales illustrating Why People Should Learn Math.

Until today it hadn't occurred to me people should learn math so they don't get their license to practice medicine yanked when they bumble a statistics and probability question in court.

Let that be a lesson to us.


low birth weight paradox (& Monty Hall)
Monty Hall, part 2
false positives
false positives, part 2
John Kay: We are likely to get probability wrong (subscription only)
Monty Hall diagram from Curious Incident

Back to main page.

---

---

Comments

After entering a comment, users can login anonymously as KtmGuest (password: guest) when prompted.
Please consider registering as a regular user.
Look here for syntax help.

WebLogForm
Title:	statistics and the law
TopicType:	WebLog
SubjectArea:	HorrorStories, StatisticsAbuse
LogDate:	200508172044