Central Limit Theorem

March 29, 2018

Pierre-Simon Laplace, 1749 -1827

The Central Limit Theorem states that the sums (or averages) of sets of N random variables will tend toward having a Normal distribution as N increases. It was the French mathematician Laplace who proved this for a number of general cases in 1810 [2]. Laplace also derived an expression for the standard deviation of the average of a set of random numbers confirming what Gauss had assumed for his derivation of the Normal distribution (more on this in the Confidence Intervals lesson).

The simplest and most remarkable example of the Central Limit Theorem is the coin toss. If a “true” coin is flipped N times, the probability of q heads occurring is given by Equation 11, which is called the Binomial Distribution.

(1) $\begin{equation*} P(q)=\frac{N!}{2^N(N-q)!q!} \end{equation*}$

Equation 11

Figure 3.10 plots a histogram of this in comparison to the Normal distribution for 6 coin tosses. There is good agreement between the two distributions, even with only 6 tosses (although the tails of the Normal distribution extend beyond the possible values of q). The French mathematician De Moivre had noticed this agreement in 1733 and used (2/πN)^1/2 e^{-(2/N)(q–N /2)2} as an approximation for the cumbersome calculation of Eq. 11 for large N. But he hadn’t generalized this to other cases.

Figure 10. Probabilities of q heads in 6 coin tosses.

Figure 3.10. Probability of q heads in 6 coin tosses.

Another example is the averaging of a random variable, x, uniformly distributed between -0.5 and 0.5 (which might be the range of uncertainty in measurement). By averaging 2, 3, and 4 of these random variables, the gradual convergence to the Normal distribution can be seen, as shown in Figure 3.11.

Figure 11. PDF's of the Averages of Uniformly Distributed Random Variables.

Figure 3.11. PDF of the averages of uniformly distributed random variables.

When the magnitude of the PDF is plotted on a linear scale it is not clear what is happening at the tails of the distribution. This can be corrected by plotting the magnitude on a logarithmic scale, so the large percentage deviation between the average of 4 uniformly distributed random variables and the Normal distribution can be seen in the tails of the distribution. This is significant in cases where the extreme values of a signal are critical to understanding the behavior of a product under test; an example is fatigue analysis.