I'm a Bayesian, you're a Bayesian, he's a Bayesian, she's a Bayesian...

I've been a Bayesian ever since I understood enough probability to know the difference between a Bayesian and a frequentist (these are two different schools of thought about probability and statistics).

Last August, I convinced Catherine that she is a Bayesian too.

Now it turns out that we're all Bayesians. This week's Economist has an article on some cognitive science research that's going to be published this year that claims to prove it.

For a little background before I whet your appetite with this idea, the core idea of Bayesian reasoning is that we reason by updating our preexisting mental 'likelihoods' of events with new information.

A simple example: when you meet a new person, you generally have a low expectation that he has chronic financial problems. Your expectation is based on your knowledge that in the general population, not many people have severe financial problems. However, if you then discover that he was laid off two months ago and had only $200 in the bank at the time, you mentally raise your estimate of his likelihood of having chronic money problems.

This much is pretty obvious. What the new research reveals is that we do perform real-time updates of our initial estimates of probabilities, in our minds, and that the probabilities we form are remarkably accurate. In short, the mathematical formalism of Bayes' formula is part of our innate mental structure, and we use it every day.

In research to be published later this year in Psychological Science, Thomas Griffiths of Brown University in Rhode Island and Joshua Tenenbaum of the Massachusetts Institute of Technology put the idea of a Bayesian brain to a quotidian test. They found that it passes with flying colours...

Dr Griffiths and Dr Tenenbaum conducted their experiment by giving individual nuggets of information to each of the participants in their study (of which they had, in an ironically frequentist way of doing things, a total of 350), and asking them to draw a general conclusion. For example, many of the participants were told the amount of money that a film had supposedly earned since its release, and asked to estimate what its total gross would be, even though they were not told for how long it had been on release so far.

Besides the returns on films, the participants were asked about things as diverse as the number of lines in a poem (given how far into the poem a single line is), the time it takes to bake a cake (given how long it has already been in the oven), and the total length of the term that would be served by an American congressman (given how long he has already been in the House of Representatives). All of these things have well-established probability distributions, and all of them, together with three other items on the list -- an individual's lifespan given his current age, the run-time of a film, and the amount of time spent on hold in a telephone queuing system -- were predicted accurately by the participants from lone pieces of data.

It turns out that we are so good at doing Bayesian analysis in our minds that Tenenbaum and Griffiths think it may be possible to determine the distributions of events in the real world by checking it against our innate Bayesian calculating machinery.

Here's a link to the article -- you need to either pay-per-view, or be a subscriber to access it.

Joshua Tenenbaum

Here's a picture of Dr. Josh Tenenbaum from MIT. He looks young enough to be my son.


Bayes statistics & false positives
does human mind use Bayesian reasoning?
Bayesian reasoning, intuition, & the cognitive unconscious
most bell curves have thick tails
ECONOMIST explanation Bayesian statistics
Bayesian certainty scale

Bayesianprobability

Back to main page.

---

Comments

After entering a comment, users can login anonymously as KtmGuest (password: guest) when prompted.
Please consider registering as a regular user.
Look here for syntax help.

---

hey!

no fair!

you beat me to it!

I have this issue of THE ECONOMIST sitting RIGHT HERE on my desk JUST WAITING TO GET THIS ARTICLE PULLED!


-- CatherineJohnson - 09 Jan 2006

---

otoh, now I don't have to scrounge for the link

-- CatherineJohnson - 09 Jan 2006

---

Here's the gif of the 4 distributions we humans apparently already have stacked up inside our heads:






-- CatherineJohnson - 09 Jan 2006

---

I personally have a picture of the Reverend Bayes inside my own head.


-- CatherineJohnson - 09 Jan 2006

---

I have this book.

It's part of my Great Unread.


-- CatherineJohnson - 09 Jan 2006

---

-- CatherineJohnson - 09 Jan 2006

---

I'm feeling visual today.

-- CatherineJohnson - 09 Jan 2006

---

Possibly because it took me almost an hour to do my Level F KUMON sheets.


-- CatherineJohnson - 09 Jan 2006

---

What was interesting to me was the notion that one can make useful and accurate predictions based in only a handful of data points, just so long as you have good 'priors':

...the Bayesian capacity to draw strong inferences from sparse data could be crucial to the way the mind perceives the world, plans actions, comprehends and learns language, reasons from correlation to causation, and even understands the goals and beliefs of other minds.

[snip]

The key to successful Bayesian reasoning is not in having an extensive, unbiased sample, which is the eternal worry of frequentists, but rather in having an appropriate “prior”, as it is known to the cognoscenti. This prior is an assumption about the way the world works—in essence, a hypothesis about reality—that can be expressed as a mathematical probability distribution of the frequency with which events of a particular magnitude happen.

The best known of these probability distributions is the “normal”, or Gaussian distribution. This has a curve similar to the cross-section of a bell, with events of middling magnitude being common, and those of small and large magnitude rare, so it is sometimes known by a third name, the bell-curve distribution. But there are also the Poisson distribution, the Erlang distribution, the power-law distribution and many even weirder ones that are not the consequence of simple mathematical equations (or, at least, of equations that mathematicians regard as simple).

With the correct prior, even a single piece of data can be used to make meaningful Bayesian predictions. By contrast frequentists, though they deal with the same probability distributions as Bayesians, make fewer prior assumptions about the distribution that applies in any particular situation. Frequentism is thus a more robust approach, but one that is not well suited to making decisions on the basis of limited information—which is something that people have to do all the time.

Actually, this is such an important point that I'm going to pull it up front...

-- CatherineJohnson - 09 Jan 2006

---

hey!

no fair!

you beat me to it!

This is the ONE TIME I beat her to the draw, and she yells at me. Phooey.

-- CarolynJohnston - 09 Jan 2006

---

I assume that "Bayesian reasoning" is what leads to my very high percentage of correct calls when someone forwards me an important email warning and I find it to be an "urban legend" already documented on snopes....

-- SusanJ - 09 Jan 2006

---

I assume that "Bayesian reasoning" is what leads to my very high percentage of correct calls when someone forwards me an important email warning and I find it to be an "urban legend" already documented on snopes....

I love it!

But which distribution would that be??

-- CatherineJohnson - 13 Jan 2006

---

No, you beat me at least one other time.....

Can't remember it....

-- CatherineJohnson - 13 Jan 2006

---

I assume that "Bayesian reasoning" is what leads to my very high percentage of correct calls when someone forwards me an important email warning and I find it to be an "urban legend" already documented on snopes....

I love it!

But which distribution would that be??

The prior would be the uniform distribution, if every other such email you've had has been an urban legend (and this is certainly true for me).

-- CarolynJohnston - 13 Jan 2006

---

What does a uniform distribution look like?

Is it just....a straight line across the graph?

-- CatherineJohnson - 14 Jan 2006

---

Exactly.

-- CarolynJohnston - 14 Jan 2006

---

oh, cool!

-- CatherineJohnson - 15 Jan 2006

---

I HAVE TO LEARN STATISTICS VERY, VERY SOON

Our district is doing 'data warehousing.'

Apparently, this means 'data mining.'

This means the administration will be able to sling the statistics along with the blah-blah.

That's not good.

-- CatherineJohnson - 15 Jan 2006

---

Start with The Cartoon Guide to Statistics.

-- GoogleMaster - 16 Jan 2006

---

I have that book!

In fact, I probably bought it 20 years ago — then when I met Carolyn she recommended it, too.

I probably better move it up on my Reading List.

-- CatherineJohnson - 16 Jan 2006

WebLogForm
Title:	I'm a Bayesian, you're a Bayesian, he's a Bayesian, she's a Bayesian...
TopicType:	WebLog
SubjectArea:	CognitiveScience
LogDate:	200601082015