The Best Statistics Question Ever

[Macrobius]

http://flowingdata.com/2011/10/28/best-statistics-question-ever/

Via Flowing Data. Plenty of discussion in the comments about possible answers -- mostly wrong.

[Ponerologist]

http://flowingdata.com/2011/10/28/best-statistics-question-ever/

Via Flowing Data. Plenty of discussion in the comments about possible answers -- mostly wrong.

I'm a bit drunk, but a few thoughts came to mind.

1) Initially I thought as there are four answers it'd be 25%.

2) Then I noticed that effectively there are three answers, as two of them are the same - which'd make it either 50% or 33.33%. The latter of which doesn't appear.

3) Then I realised I was very confused. ;)

I will blame it on drunkeness to try to spare my blushes. But the truth is that were I sober I'd probably get it wrong. I once got 13% in a stats test at university.

[Jimbo Gomez]

I think Luke here is correct.

Luke — November 1, 2011 at 10:37 am
Read the question again – it never implies the answer is one of the multiple choice options, if you rephrase it then it’s more understandable.
Picture there being 3 light bulbs and 4 doors, 2 doors lead to the light bulb on the left, and the other 2 doors lead to 1 of the light bulbs each. If the bulbs light up randomly, what’s the probability of walking through a door into a lit up room? If the left one lights up you’ll be right 50% of the time, if the middle one is on then 25% of the time, and if its the right one then 25% of the time as well. Each of these light bulbs randomly will light up 33% of the time, therefore -
[33% * 50%] + [33% * 25%] + [33% * 25%] = Probability of being right = 33%
And there we go, the answer to this question is 33%.
The question is not multiple choice, the problem that the question is about is. It’s not a paradox, there is obviously an answer, and it’s clear as day once you think about it.

[Angler]

Okay, I'll take a stab at it.

First assume that the percentage after each lettered answer choice refers to the probability the question is asking for. In other words, those percentages don't refer to something unknown and unstated in the problem.

Say we choose A at random. Is 25% the correct answer? No. Because if it were, then choosing D at random would also give the correct answer of 25%. That's a contradiction, since the probability of choosing A or D at random is 50%.

Similarly, D cannot be the correct answer.

Now say we choose B at random. We had a 25% chance of doing this. Can B then be the correct answer? No, because choice B says we have a 50% of choosing the correct answer.

C can be ruled out at a glance.

Thus, there are no correct answers in those four choices. The probability of choosing the correct one at random is 0%.

[Baron_Corvo]

I think the key to it is where it says, "Try not to think too hard". In other words, it's one of those "flash of insight" types of questions.

Still waiting for such a flash :)

[Ixtab]

The answer is 25%. You have a 25% chance of selecting A or D, even though both answers have the same value of 25%. The odds of picking B (50%) are still 25%.

[Macrobius]

I've given my own answer at the Original Dissent 'home away from home'.

http://previousdissent.com/forums/showthread.php?15823-The-Best-Statistics-Question-Ever&p=21380#post21380

However, it is more fun to try to answer for yourself, or read the confused ramblings of the 625+ comments at the OP link.

[Stanley]

My take is that there is no right answer. The question is entirely self-referential, and so contains a paradox that is impossible to resolve, on the lines of

A. Statement B is true.
B. Statement A is false.

[Dapper Old Gent]

I agree with Angler, depending that only one answer can be correct. In some schools they say "best" answer as they may all be correct.

[Ixtab]

Is this a form of Russell's paradox?

http://en.wikipedia.org/wiki/Russell%27s_paradox

[Ixtab]

I agree with Angler, depending that only one answer can be correct. In some schools they say "best" answer as they may all be correct.
That can't be correct because 0% isn't listed. The chance of getting 0% by choosing one of the four choices is 0%, not because it is the correct answer, but because it is excluded from the possible answers; and if you include answers not listed, there is no way to evaluate the probability of arriving at the correct answer.

[Ixtab]

Also, if 0% is ever correct, then the chance of randomly selecting 0% is non-zero, which is a contradiction.

[Macrobius]

The question is an example of a discrete probability distribution. You are allowed four choices (A and D are distinct choices but have equal outcomes of 0.25). The problem is of course the self-referential or consistency aspect of the problem.

Let's look at 'answer you pick' vs. 'answer that is right' and try to make up a constraint on what a consistent answer is.

If you pick 0.25 (i.e., choose either A or D), then 2 x (0.25) must be the correct answer. The fact that input = output is out of bounds here means that is an inconsistent choice.

If you pick 0.5, then 0.5 must be the output.

If you pick 0.6, then 0.6 must be the output.

Thus, there are only two consistent answers -- B and C. Neither can be the sole answer (because neither is 100%), and neither can be an answer for a uniform discrete distribution that picks A-D with probability 1/4. The only possible solution is to allow 'random' to mean choosing the answers with a *non-uniform discrete probability distribution*

http://en.wikipedia.org/wiki/Uniform_distribution_%28discrete%29 -- the problem statement allows that.

[Ixtab]

The question seems to assume a uniform distribution. You can't randomly select one answer more than the other. If you could choose your own probability distribution, then any answer might be correct. But you are instructed to select only one answer.

If you pick 0.25 (i.e., choose either A or D), then 2 x (0.25) must be the correct answer. But you have a 25% chance of selecting 50%. Whatever answer is "correct", you have a 25% of selecting it at random because the question instructs you to select "an (= one) answer" out of four allowable answers. Even if all the answers were 25%, you'd still have only a 25% chance of selecting any one of those answers. It does not matter what answer you select, but the probability of selecting it, which can only be 1/4.

[Macrobius]

The question seems to assume a uniform distribution. You can't randomly select one answer more than the other. If you could choose your own probability distribution, then any answer might be correct. But you are instructed to select only one answer.


I understand that in common parlance, 'random' implies 'uniform'. However, we are specifically informed this is a statistics question, and so I assume that since most statisticians are well-informed about the possible existence, if not ubiquity, of non-uniform random variates, that a discrete but non-random distribution is a reasonable answer.

Here is a classic book on the subject, for discrete and continuous variables: http://www.nrbook.com/devroye/


But you have a 25% chance of selecting 50%. Whatever answer is "correct", you have a 25% of selecting it at random because the question instructs you to select "an (= one) answer" out of four allowable answers. Even if all the answers were 25%, you'd still have only a 25% chance of selecting any one of those answers. It does not matter what answer you select, but the probability of selecting it, which can only be 1/4.

I don't think this is true. The 'correct' answer is defined as the correct expectation value -- which is just a number, not a label, like A or D.

Your observation would be correct if the correct answer were arbitrary. For example, if the test had for identical answers, all 25%, you would be arguing any one of them is correct because the distribution is uniform (therefore 25%), and thus choosing any answer is correct.

I would be arguing, in that case, the expectation of choosing the correct answer is 100%, since all of them are identical and you say correct, only since none of them are 100%, all of them are false, and the expectation is zero, therefore the problem is inconsistent.

However, this problem is not like that. Two of the answers are consistent. The question is whether either of them is the expectation in the sense of the question!

[Angler]

I agree with Angler, depending that only one answer can be correct. In some schools they say "best" answer as they may all be correct.That can't be correct because 0% isn't listed. The chance of getting 0% by choosing one of the four choices is 0%, not because it is the correct answer, but because it is excluded from the possible answers; and if you include answers not listed, there is no way to evaluate the probability of arriving at the correct answer.Ah, but I already noted that the correct answer of 0% isn't among the four answer choices.

No matter which choice you pick from those four answers, it will be wrong. Therefore, your chances of picking the "correct" answer are 0%. 0% does not need to be one of the four answer choices in order to recognize this.

[Macrobius]

Ah, but I already noted that the correct answer of 0% isn't among the four answer choices.

No matter which choice you pick from those four answers, it will be wrong. Therefore, your chances of picking the "correct" answer are 0%. 0% does not need to be one of the four answer choices in order to recognize this.

This is only true if you assume that 'random' *must* imply a 25% probability of picking each of A-D respectively. That is, if you assume 'random' implies 'uniform'. The problem provably has no correct answer [assuming the usual convention that in a multiple choice test you may only choose one, of which at least one is correct], if that is the case.

I deny the problem requires this assumption, however.

IB hit it on the head:


If you could choose your own probability distribution, then any answer might be correct. But you are instructed to select only one answer.


Not quite 'any answer', however. You can prove some won't work, and you can attain others. Among those you can attain, there is a best.

It is indeed a key question here, whether you can choose your own probability distribution. If you can, then (C) is the clear best choice. When answering the test question, you should throw some dice and choose it 60% of the time, and your chance of getting that question correct on the test will be 60%. Picking the maximum answer is the very best you can do, if you can arrange the distribution to suit your own victory (why not??). Answering the question deterministically is 100% FAIL. The best you can do is the game-theoretic answer of selecting randomly in a non-uniform way (i.e., with weights chosen by game theory, not uniform -- and you jigger them to get the 'best answer', which is what you are supposed to do on a test, if there is no 100% answer, as someone noted above).

This is really a question in Statistical Mechanics: if you *don't* get to pick the distribution [Nature does, and Nature plays the odds], then the only reasonable victory strategy is to go with maximum entropy and pick (B). If you do get to pick it, then of course you should pick (C) and jigger the distribution to make your choice a winner. It is consistent and the highest value, but only sometimes if Nature gets to pick (namely, when your jiggering makes it so and hardly ever otherwise), not on the average. If you have no information about what the average is, then maximum entropy tells you what will mostly happen, so far as the space of non-uniform discrete random distributions is concerned.

If you have to pass the test, pick (B), or pick (C), if you are feeling lucky, punk.

[Ixtab]

But we aren't provided with any information aside from that contained within the answers. What is "correct" is not defined, nor can it be deduced from the question itself. The correct answer can only depend on what assumptions we make about the question. If we assume that there is only one correct answer to the question, and it is one of the four listed answers, then the answer can only be 25%; for we have no reason to suppose that just because labels A and D represent the same value, that if one is true, then the other must also be true. For all we know the test-maker might have decided that A is correct and no other answer.

[Angler]

This is only true if you assume that 'random' *must* imply a 25% probability of picking each of A-D respectively. That is, if you assume 'random' implies 'uniform'. The problem provably has no correct answer [assuming the usual convention that in a multiple choice test you may only choose one, of which at least one is correct], if that is the case.

I deny the problem requires this assumption, however.I don't see it as an assumption, but as a necessity. If a person picks one of four letters/answers at random, how can he be more likely to pick one than another? What could be special about any of the letters/answers that would make it (or them) more or less likely to be picked?

I think of it as picking one of four marbles out of a box while wearing a blindfold. The marbles are identical in every respect -- size, texture, etc. -- except for color. The four marbles are blue, red, yellow, and green. This situation is described by a uniform distribution, and each color marble has a 25% chance of being picked. How is this not mathematically equivalent to choosing one of four answer choices at random?

[Ixtab]

But we aren't provided with any information aside from that contained within the answers. What is "correct" is not defined, nor can it be deduced from the question itself. The correct answer can only depend on what assumptions we make about the question. If we assume that there is only one correct answer to the question, and it is one of the four listed answers, then the answer can only be 25%; for we have no reason to suppose that just because labels A and D are the same, that if one is true, then the other must also be true. The test-maker could have decided that A is correct and no other answer.
Also assuming equal probability, which, as I think Angler has shown, is a justified assumption.

[Macrobius]

I don't see it as an assumption, but as a necessity. If a person picks one of four letters/answers at random, how can he be more likely to pick one than another? What could be special about any of the letters/answers that would make it (or them) more or less likely to be picked?

Statisticians consider the possibility of 'all discrete distributions', which allows for you to 'randomly choose' not only 1/4-1/4-1/4-1/4, but a-b-c-d, where those are random variables that add up to unity. For example 1/2-1/4-1/8-1/8, or even 1-0-0-0. Picking only 'A' for every question is just as random a strategy as spreading your choices around evenly. It has lower multiplicity (entropy) than the other ways, because there is only one way to do it. But it is still validly random.

Like I said, this is a common misunderstanding of how Statisticians think, on the part of other Scientists (and certainly Laymen).


I think of it as picking one of four marbles out of a box while wearing a blindfold. The marbles are identical in every respect -- size, texture, etc. -- except for color. The four marbles are blue, red, yellow, and green. This situation is described by a uniform distribution, and each color marble has a 25% chance of being picked. How is this not mathematically equivalent to choosing one of four answer choices at random?

Ah, but it is entirely possible the number of marbles is not equal to begin with, and you can still pick them at random. Your 'and each color marble has a 25% chance of being picked' is merely an additional hypothesis about initial equality of colour numerosity, that you care to introduce. If you do so in the context of this problem, then obviously you render the problem statement inconsistent. Test taking strategy alone, and your sense of survival, should suggest you might want to do something that leaves you a non-zero, yet 'living' option, at the end of the day, and non-equality is provably the only way out, in this case. Then again, maybe you are an evolutionary dead end who wants to die for a principle of uniformity. Many Democrats do.

Pick any Nigger at random. They are just as likely to 'succeed' as Whites. It is provable, and nothing else can possibly be truer. I can't even conceptualise the contrary.

[Gun]

This is not a paradox and there is no correct answer. Its just a simple case of the incorrect use of language.▲

The problem with the exercise is that it refers to a question that hasn't been asked. The grammatical formulation: "if you choose an answer to this question, what is the chance you will be correct?" Asks the chance of correctness for a question that hasn't been formulated. Which question am I to be correct or incorrect in answering? Or else it is self-referential and the question is that is being referred to as "this question" is the question "what is the chance you will be correct?" In which case the question is meaningless because this particular type of question refers to another question and cannot furnish a criteria in itself for correctness of incorrectness. The question "what is the chance you will be correct" is a question that implies a previous question that has criteria for being right or wrong. The question requires a previous question which there is a separate set of answers which could be broken down by percentages. It is an unhappy utterance in the game of asking questions, and the answer to the question might as well be "purple monkey dishwasher."

Or else the question could be asked: if you are given a set of four numbers, 25, 25, 50 and 60, what is the chance that you will pick a particular number in the set, and if so is it the same as a number in that set? In which case, the number 25 can't be in the same set twice. One could imagine someone saying "who said anything about sets?" But, then, I would challenge them to show me another way of actually formulating the question that leads to the problem. "Given four options of which only one is correct, what is the percentage chance that a person will randomly pick the right answer?" The answer is 25. Even if that answer shows up again, it is still 25 and we simply have a case of having two right answers. This is not impossible by any means for the purely mechanical process of evaluating a multiple choice test. There is no way to reformulate this question in such a manner as to reproduce the original problem, and the way it is formulated is an unhappy utterance that is, strictly speaking, meanigless. Case closed.

[Macrobius]

This is not a paradox and there is no correct answer. Its just a simple case of the incorrect use of language.

The problem with the exercise is that it refers to a question that hasn't been asked. The grammatical formulation: "if you choose an answer to this question, what is the chance you will be correct?" Asks the chance of correctness for a question that hasn't been formulated. Which question am I to be correct or incorrect in answering? Or else it is self-referential and the question is that is being referred to as "this question" is the question "what is the chance you will be correct?" In which case the question is meaningless because this particular type of question refers to another question and cannot furnish a criteria in itself for correctness of incorrectness. The question "what is the chance you will be correct" is a question that implies a previous question that has criteria for being right or wrong. The question requires a previous question which there is a separate set of answers which could be broken down by percentages. It is an unhappy utterance in the game of asking questions, and the answer to the question might as well be "purple monkey dishwasher."

Bayesians tend to believe, if there are two outcomes and you have no information about either, the odds of your being correct/not correct with regard to that outcome are 50-50 a priori. That is, your subjective estimate of the prior distribution is indifferent between the two possibilities.

That is, the states that matter are not the number of questions (4), but the possible states of reality (you are correct about the expectation of the outcome, you are not correct about that). In this case, the decider has four possible choices offered to the test-taker, and there are three possible states of nature (assuming the question is consistent, since only three distinct answers are offered to choose from -- and however many states of nature you like if it is not consistent, of course, since in that case the answer might be anything).

There is nothing *inconsistent* with the idea that the different non-uniform random choice distributions represent possible worlds, and that the average randomly chosen possible world returns answer (B), which is the correct choice for the test taker, since it happens to be the answer to the question.

[Angler]

Statisticians consider the possibility of 'all discrete distributions', which allows for you to 'randomly choose' not only 1/4-1/4-1/4-1/4, but a-b-c-d, where those are random variables that add up to unity. For example 1/2-1/4-1/8-1/8, or even 1-0-0-0. Picking only 'A' for every question is just as random a strategy as spreading your choices around evenly.

Like I said, this is a common misunderstanding of how Statisticians think, on the part of other Scientists (and certainly Laymen).It can certainly make sense when considering some problems to consider the possibility that a non-uniform distribution may apply. But this isn't one of those problems.

If you roll an ordinary six-sided die, aren't the possible outcomes necessarily described by a uniform distribution? With the exception of the number of possible outcomes, how is this roll of the die mathematically different from picking one of four letters on a piece of paper completely at random?

Ah, but it is entirely possible the number of marbles is not equal to begin with, and you can still pick them at random.According to the definition of the problem, that is not possible.

Your 'and each color marble has a 25% chance of being picked' is merely an additional hypothesis about initial equality of colour numerosity, that you care to introduce.No, it follows from how the problem was defined. If the problem states there are four marbles that are identical except for their four different colors, then there is most certainly a 25% chance that any given color marble will be picked. You don't get to redefine the problem, because that would mean you were solving a different problem than the one originally posed.

You can even test this for yourself if you care to. Or just flip an ordinary coin instead. After a few hundred trials, is there any way that the probability distribution won't turn out to be a uniform 50-50? Could it turn out to be 30-70 or 40-60? No, it could not. The larger the number of trials, the closer you'll get to 50% heads, 50% tails.

If you do so in the context of this problem, then obviously you render the problem statement inconsistent.The problem is necessarily inconsistent as long as one operates on the assumption I mentioned in my first post on this thread. No assumption is needed about uniform probability distributions, since the distribution here is necessarily uniform if one of the answer choices is truly picked at random.

Test taking strategy alone, and your sense of survival, should suggest you might want to do something that leaves you a non-zero, yet 'living' option, at the end of the day, and non-equality is provably the only way out, in this case.There's no need for test-taking strategy here or a "way out" of anything.

Then again, maybe you are an evolutionary dead end who wants to die for a principle of uniformity. Many Democrats do.:confused:

Pick any Nigger at random. They are just as likely to 'succeed' as Whites. It is provable, and nothing else can possibly be truer. I can't even conceptualise the contrary.That depends on how "success" is defined. Money? Happiness?

But let's take something more concrete. Pick any nigger at random. Is he just as likely to have committed rape as any randomly chosen white person? The answer is no. He's more likely. We know this because of the available statistical data.

[Gun]

Bayesians tend to believe, if there are two outcomes and you have no information about either, the odds of your being correct/not correct with regard to that outcome are 50-50 a priori. That is, your subjective estimate of the prior distribution is indifferent between the two possibilities.

That is, the states that matter are not the number of questions (4), but the possible states of reality (you are correct about the expectation of the outcome, you are not correct about that). In this case, the decider has four possible choices offered to the test-taker, and there are three possible states of nature (assuming the question is consistent, since only three distinct answers are offered to choose from -- and however many states of nature you like if it is not consistent, of course, since in that case the answer might be anything).

There is nothing *inconsistent* with the idea that the different non-uniform random choice distributions represent possible worlds, and that the average randomly chosen possible world returns answer (B), which is the correct choice for the test taker, since it happens to be the answer to the question.

I just edited my original post with some additional thoughts I came up with just now, and I think they relate to what you're saying here.

[Macrobius]

No assumption is needed about uniform probability distributions, since the distribution here is necessarily uniform if one of the answer choices is truly picked at random.

Saying so doesn't make it so. I've noticed you often assert things that are not necessarily true, as if they were so, however, so I'm not surprised you push a non-essential assumption as a necessary truth here as well. The non-uniform odds are in favour of it, in this case.

You can even test this for yourself if you care to. Or just flip an ordinary coin instead. After a few hundred trials, is there any way that the probability distribution won't turn out to be a uniform 50-50? Could it turn out to be 30-70 or 40-60? No, it could not.

Quite easily. Ordinary coins are frequently biased, contrary to your assumption they are not. Provably so, using empirical evidence. They come up 'heads' more frequently, in fact, for a typical coin, with some small bias. Convergence to the mean doesn't restrict the mean to exactly 0.500000.... and so on forever amen -- it can be any mean between 0 and 1 in fact, with some 'non-uniform distribution', and often is. Coins are only fair to within a certain tolerance, which varies from coin to coin, and in principle a 6 sigma defective coin could be quite, quite off your absurd notion of necessary perfect fairness for all coinage whatsoever.

I'm beginning to understand why you are an atheist.... and on what evidence.


There's no need for test-taking strategy here or a "way out" of anything.


So what is your final answer?

[Gun]

Saying so doesn't make it so. I've noticed you often assert things are are not necessarily true if if they were so, however, so I'm not surprised you push a non-essential assumption as a necessary truth here as well.



So what is your final answer?

In this case, in the quest for Sophia's hand, she says "stop fighting, fellas: you can both marry me!!" But in this case she also turns out to be a princess who turns into a frog. There is no answer to a meaningless question. :)

[Macrobius]

... "Given four options of which only one is correct, what is the percentage chance that a person will randomly pick the right answer?" The answer is 25.


I think you are mistaking the choice made, and the resulting state of nature. It is certainly reasonable to distinguish:

p(Correct Answer is chosen|A is chosen) = p(A is chosen|Correct answer is chosen)*p(Correct answer is chosen)/p(A is chosen)

You can write down 4 equations (one for each choice), which depend on two states of nature ... 'the correct answer is chosen', x, and 'the correct answer is not chosen', 1-x, as well as the non-uniform frequencies for choosing each question, 'the probability the answer A is chosen' (and B,C, D similarly).

The denominators may be expanded p(A is chosen) = p(A is chosen|A is correct)*x + p(A is chosen | A is not correct)*(1-x)

If you set p(A is chosen) = p(B is chosen) = p(C is chosen) etc. all equal to 0.25, you may indeed show that the set of equations is inconsistent ... [b]which is a damn good reason not to do that if you wish to solve the problem. ;)

If you do find a consistent solution, it won't be one where x=0.25, contrary to what you assert. Also, there is a solution, again contrary to your assertion there is no such (minus your assumption which makes that trivially true).

[Baron_Corvo]

I'll go with the "Agatha Christie" option here; the answer is 60%, because at first sight it couldn't be.

I'll read the thread thoroughly later. Have to admit though that the self-referentiality Macrobius refers to jammed up my brain when I looked at it last night and I couldn't go any further.

[Transcendentally Challenged]

The answer is 0%.

What options are given is irrelevant, since its phrased to make any attempt at answering meaningless. One should stop after reading and ask himself - now, what exactly am I being asked here?

Question is a request for information. Here despite the "?" in the end, there's no question.

[Jimbo Gomez]

That can't be correct because 0% isn't listed. The chance of getting 0% by choosing one of the four choices is 0%, not because it is the correct answer, but because it is excluded from the possible answers; and if you include answers not listed, there is no way to evaluate the probability of arriving at the correct answer.


The question speaks about picking an answer at random, not about picking an answer at random FROM THE LIST BELOW. So it isn't 25%. I'd like to change my answer to 1/infinity, after having thought about it.

[Transcendentally Challenged]

The question speaks about picking an answer at random, not about picking an answer at random FROM THE LIST BELOW. So it isn't 25%. I'd like to change my answer to 1/infinity, after having thought about it.

An answer to what? What is the question?

[Jimbo Gomez]

An answer to what? What is the question?


That self-referential paradox is the question.

[Transcendentally Challenged]

That self-referential paradox is the question.

Self-referred question is a logical impossibility. You can't ask about something which doesn't exist at the point when you begin talking.

[Mackie]

Whats the question again? The sentence does not describe the question i am supposed to be answering to, only the method of execution and possible outcomes.

If I forego that odd bit, the chance that ill be right is 50%, either im right or im not.
[edit: i have no predisposition for either so by my logic it could be one as much as the other]

[Errigal]

Self-referred question is a logical impossibility. You can't ask about something which doesn't exist at the point when you begin talking.

Read the question, it clearly forms a loop on itself so that an answer of 25% or 50% are both possibilities.

[Transcendentally Challenged]

Read the question, it clearly forms a loop on itself so that an answer of 25% or 50% are both possibilities.

I read the question and from my opinion, its not a question, its a linguistical trap, when something is phrased like a question yet isn't one.

[Errigal]

I read the question and from my opinion, its not a question, its a linguistical trap, when something is phrased like a question yet isn't one.

Yes it's a linguistic trap.

[Ixtab]

The question speaks about picking an answer at random, not about picking an answer at random FROM THE LIST BELOW. So it isn't 25%. I'd like to change my answer to 1/infinity, after having thought about it.
I think it should be assumed that a question on a blackboard with four answers listed A to D is a multiple choice question. If we get rid of the multiple choice answers, the question becomes: What is the chance of randomly picking a correct number? Which is a meaningless question. It only becomes meaningful, though perhaps still unanswerable, with the multiple choices.

[Macrobius]

The question cannot be asking to 'pick a number' at random, from the continuum [0,1]. Four discrete answers comprise a 'set of measure zero' on the continuum and the chance of picking any of them at all 'at random' is most precisely zero.

However, I don't think a related multiple choice question would be unanswerable or paradoxical. Suppose there were only two answers instead of four:

(A) 50%
(B) 0%

I think most of us would agree that answering the question at random, with equal probability (1/2 the time A, 1/2 the time B), would select (A) 50% of the time, and thus if one were to choose random, equally-weighted guessing, then there would indeed be one correct answer -- which would be selected at random 1/2 the time, by a person hypothetically choosing to guess rather than score highly on the test. It should be selected 100% of the time by anyone not auditing the course. ;)

Of course, a student taking a test and wishing to pass the course should select (A). The hypothetical statement allows you to evaluate the 'what if' of random guessing, and then to *choose* to answer the question correctly and pass the course. In an English course, you are permitted to write a story about what it would be like to be a monkey, without imitating a monkey while writing it and putting random marks on your paper. You are allowed to answer the question hypothetically, with a human, coherent account of what it would be like.

[Macrobius]

Further discussion at Previous Dissent:

http://previousdissent.com/forums/showthread.php?15823-The-Best-Statistics-Question-Ever

More discussion on the Phora thread and some final comments.

We discussed a similar puzzle previously (the 'two envelopes paradox'): http://en.wikipedia.org/wiki/Two_envelopes_problem

The key there is you don't know what's in the envelopes, nor the probability distribution created by the 'arranger': http://www.thephora.net/forum/showthread.php?t=63394

My solution is to invent a parameter space or 'configuration space' (a,b,c,d) of non-uniform probabilities to select the answers to the question -- which represent the proportion of answering each way, randomly. Odds are *not* the same as probabilities. Nature will certainly select 'blindly' from the possible configurations, subject to constraints such as a+b+c+d=1. Those who argue a=b=c=d=0.25 is necessary for Nature, are barking up the wrong tree. You can get the same result as you want, using Statistical Mechanics, by considering the ensemble of possible configurations, and that the 'equipartition' result you seem to want happens because those configurations are the Most Likely (i.e.,, the likelihood has a Mode there). This is a consequence of Entropy, not of Nature being forced to be fair.

A bit of thought shows that selecting (B) or (C) with less than 100% probability are the only feasible solutions. The parameter space (of non-uniform odds for picking A-D) is 4 dimensional. The constraint a+b+c+d=1 restricts answers to a 3d hyperplane of that 4-space. Within that 3-d hyperplane, the only feasible regions are to select sharp values of (exclusively) b=0.5 or c=0.6. This selects two non-overlapping subspaces. You can prove they are non-overlapping because if b=0.5 and c=0.6 simultaneously, then the sum-to-one constraint is violated.

The notion that one is playing against Nature won't pan out -- nature is blind, and the expectation over the whole 3d hyperplane is calculable, but not one of the answers given (proof left as exercise for the reader). However, there is no reason to suspect the problem is posed for Nature to pick the non-uniform distribution. A better model for test taking is the 'Rational Expectations' model -- some students may in fact be quite inferior, but they shouldn't dictate *your* distribution. On the other hand, it is reasonable to let the set of Rational students do so -- namely, those versed in Statistics, and optimising their test scores. Thus, we only need consider the feasible portions of parameter space, and only those that optimise a rational agent's (test taker's) score. After all, all test takers are enjoined to answer the test optimally!

An interesting economic formulation is to charge a fee to play a game, with say 1000 players, and let them select (A)-(D) with any 'trading strategy they like', but with payoffs only if they pick the average expectation of the group within some tolerance epsilon = 5%, say. (This move should satisfy the Frequentists, who will accept the notion of 'random' only in the limit of N players approaches Infinity anyway!). The price of finite N is you have to allow a tolerance epsilon (small enough to separate victory conditions). This is a small price, and you can always insist that the choice of N and epsilon get large and small, respectively, and that the 'correct' answer emerge in that limit.

Our economic game is similar to the stock market then, where the 'group assessment' (of what the non-uniform distribution of outcomes is) determines the 'price'. Success means guess your own success correctly. Our economic system solves problems like this every day!

Thus, from a game theory perspective, you may select one of two subspaces, and those are the only strategies. If both players cooperation (non-zero sum payoff), then the 'higher equilibrium' solution is possible -- all rational agents recognise their mutual interest in excluding the X = 0.5 equilibrium, and all of them select choice (C) *exactly* 60% of the time, using a random process to do so. *whew*.

If you assume that rational agents are non-cooperative, then you do worse. If they cooperate, the expected value of the game is 0.6. If they don't, but still behave rationally, they can get at least 0.5. The solution for a zero sum game is to play the (B) strategy and (c) strategy in an odds ratio of 6:5. That is, get a random number and 6/11ths of the time play (B) 50% and spread your play randomly among the other choices 50%. 5/11ths of the time play (C) 60%, and spread your play randomly among the other choices 40% of the time. This is the best you can do. The payoff of this non-cooperative strategy is 6/11ths ([6(0.5)+5(0.6)]/11 = 6/11ths). Sadly, that is not an option so optimal Zero Sum play leads to 0% success for all players.

Thus, summarising:

Play against irrational agents (the usual case)... you might as well select (C) exactly 60% of the time at random for the contrafactual, and thus answer (C) 100% of the time for the test.

Play against nature ... there is no consistent answer.

Cooperative play with rational agents maximizing their joint utility ... everyone answers (C)

Zero-sum competitive play among rational agents maximizing their individual utility.. everyone loses all the time.

Another important consideration... http://en.wikipedia.org/wiki/Law_of_total_expectation

[Baron_Corvo]

Here's an answer I got from a different board;

"This isn't a well-defined probability question, hence its answer is indeterminate. Note that this does not mean that the answer is 0% (sorry quimbly). There is no answer.

The problem is one of metalanguages. You cannot define a set using only elements of the set. This question is the equivalent of Russell's Paradox. It demonstrates the importance of defining your sets carefully.

Formally: what is the probability space over which the probabilities are being calculated? It is the set of multiple choice answers that can be chosen. How is the probability calculated from this probability space? By counting the number of correct answers. How many answers are correct? It depends on the number of multiple choice answers and what they say. Hence we are defining the probability space using the definitions of the probabilities that are comprising the probability space. Hence paradox."

[Macrobius]

I think that is right -- that doesn't mean humans are unable to solve the problem by playing a game or using economics, however. Such paradoxes are in fact solved on a daily basis, at poker or in the stock market. The value of a poker hand or a security is determined by the 'game context' and not a fixed prior distribution. The answer isn't unique -- it depends on the properties of the game as well -- but for a given game, there are various equilibria. And after all, test-taking is a well known game we have all played in our youths...

However, mathematics cannot give an answer (unless you import an assumption about the distribution -- some of those assumptions lead to problems that are not well posed.) My argument is that tests still have best answers, even if they are formulated in terms of paradoxes. After all, you still have to do something, even if it is refusing to answer the question.

In re Russell: You are in a desert, and come across two vessels, but containing an identical clear brown liquid, one of which contains in fact wholesome tea, and the other a deadly poison that looks and tastes exactly like tea. One is shaped like a teapot and says 'I am not a teapot' on it. The other, shaped like a vial, is labeled 'I am a teapot ... the not-teapot contains poison'. If you drink neither, you will die of thirst ....

If you have no prior information of any worth at all, then the answer is 50-50, or (B) in our problem above. As Piglet said, 'and what do you think you will answer yourself, Pooh?'

[007]

Depends which is the correct answer. If 25% is the correct answer you've got 2 chances out of four to pick it. If the correct answer is 50% or 60% you've got one chance in four to pick it.

Edited to add; So you should take the average as your answer. That number isn't one of the choices listed so if you must choose one of the four then obviously D. 25 % is a mistake, you should cross it out, write in the average and then circle A as your answer