Central Limit Theorem

Back to: Random Testing

The Central Limit Theorem states that the sums (or averages) of sets of N random variables will tend toward having a Normal distribution as N increases. It was the French mathematician Laplace who proved this for a number of general cases in 1810 [2]. Laplace also derived an expression for the standard deviation of the average of a set of random numbers confirming what Gauss had assumed for his derivation of the Normal distribution (more on this in the Confidence Intervals lesson).

The simplest and most remarkable example of the Central Limit Theorem is the coin toss. If a “true” coin is flipped N times, the probability of q heads occurring is given by

(which is called the Binomial Distribution). Fig. 10 plots a histogram of this in comparison to the Normal distribution for 6 coin tosses. There is good agreement between the two distributions, even with only 6 tosses (although the tails of the Normal distribution extend beyond the possible values of q). The French mathematician De Moivre had noticed this agreement in 1733, and used  (2/πN)^1/2 e^{-(2/N)(q–N /2)2}  as an approximation for the cumbersome calculation of Eq. 11 for large N. But he hadn’t generalized this to other cases.

 

Another example is the averaging of a random variable, x, uniformly distributed between -0.5 and 0.5 (which might be the range of uncertainty in a measurement). By averaging 2, 3, and 4 of these random variables, the gradual convergence to the Normal distribution can be seen, as shown in Fig. 11.

 

When the magnitude of the PDF is plotted on a linear scale it is not clear what is happening at the tails of the distribution. This can be corrected by plotting the magnitude on a logarithmic scale, so the large percentage deviation between the average of 4 uniformly distributed  random variables and the Normal distribution can be seen in the tails of the distribution. This is significant in cases where the extreme values of a signal are critical to understanding the behavior of a product under test; an example is fatigue analysis.