Principle of indifference

The Principle of indifference is a rule for assigning epistemic probabilities amongst n mutually exclusive possibilities, where n is a positive integer. It states that if there is no reason to favor a particular possibility, then each possibility is to be assigned a probability of 1/n. Naturally, this rule is meaningless to those who espouse the frequency interpretation of probability, for whom probabilities are estimated from observations, rather than assigned to reflect a state of information.

Examples

The textbook examples for the application of the principle of indifference are coins, dice, and cards. Another example of the use of the principle of indifference can be found in the derivation of the partition function.

Coins

A symmetric coin has two sides, arbitrarily labelled "heads" and "tails". We toss the coin; applying the principle of indifference, we assign each of the possible outcomes a probability of 1/2.

Dice

A symmetric die has n faces, arbitrarily labelled from 1 to n (n is usually 6, but not always). We toss the die; applying the principle of indifference, we assign each of the possible outcomes a probability of 1/n.

Cards

A standard deck contains 52 cards, each given a unique label in an arbitrary fashion, i.e. arbitrarily ordered. We draw a card from the deck; applying the principle of indifference, we assign each of the possible outcomes a probability of 1/52.

This example, more than the others, shows the difficulty of actually applying the principle of indifference in real situations. What we really mean by the phrase "arbitrarily ordered" is simply that we don't have any information that would lead us to favor a particular card. In actual practice, this is rarely the case: a new deck of cards is certainly not in arbitrary order, and neither is a deck immediately after a hand of cards. In practice, we therefore shuffle the cards; this does not destroy the information we have, but instead (hopefully) renders our information practically unusable, although it is still usable in principle. In fact, some expert blackjack players can track aces through the deck; for them, the condition for applying the principle of indifference is not satisfied.

History of the Principle of indifference

The original writers on probability, primarily Jacob Bernoulli and Pierre Simon Laplace, considered the principle of indifference to be intuitively obvious and did not bother to give it a name. Laplace wrote:

The theory of chance consists in reducing all the events of the same kind to a certain number of cases equally possible, that is to say, to such as we may be equally undecided about in regard to their existence, and in determining the number of cases favorable to the event whose probability is sought. The ratio of this number to that of all the cases possible is the measure of this probability, which is thus simply a fraction whose numerator is the number of favorable cases and whose denominator is the number of all the cases possible.

These earlier writers, Laplace in particular, naively generalized the principle of indifference to the case of continuous parameters, giving the so-called "uniform prior probability distribution", a function which is constant over all real numbers. He used this function to express a complete lack of knowledge as to the value of a parameter.

The Principle of insufficient reason was its first name, given to it by later writers, possibly as a play on Leibniz's Principle of Sufficient Reason. These later writers (George Boole, John Venn, and others) objected to the use of the uniform prior for two reasons. The first reason is that the constant function is not normalizable, and thus is not a proper probability distribution. The second reason is that complete lack of knowledge about the value of a parameter implies complete lack of knowledge about the square of the parameter, or the cube. But a uniform prior for a given parameter does not imply a uniform prior for the square or the cube of the parameter. (The problem applies not just for powers but for any function of the parameter.) Thus, the principle of indifference, naively applied in the continuous case, is logically inconsistent.

The "Principle of insufficient reason" was renamed the "Principle of Indifference" by the economist John Maynard Keynes, who was careful to note that it applies only when there is no knowledge indicating unequal probabilities. It turns out to be a special case of the Principle of maximum entropy, and can be given a deeper logical justification by the Principle of transformation groups.