Exponential Distributions

Exponential distributions are typically used to determine probabilities for the waiting time until a success occurs, when the mean rate of success per unit of time is known. In this respect, the exponential distribution is related to the Poisson distribution. Exponential distributions are always defined on the interval $[0,\infty)$.

The Formulas

If $X$ is exponentially distributed over the interval $[0,\infty)$, then the following formulas will apply.

\begin{align} f(x) &= \lambda e^{-\lambda x} \\ F_X (x) &= 1 - e^{-\lambda x} \\ M(t) &= \dfrac{\lambda}{\lambda - t} \\ E(X) &= \dfrac{1}{\lambda} \\ Var(X) &= \dfrac{1}{\lambda^2} \end{align}

Time Until Failure

Suppose that an electronic component has a mean life of 500 hours. What is the probability that the component will last at least 600 hours?

The random variable in this problem is the time, and in particular that a certain event (the failure of the component) will not occur until at least 600 hours have passed. That is, we want to find the quantity $P(T > 600)$.

Since the mean life of the component is 500 hours, and the "success" in the problem is the event that the component fails, the mean rate of success per unit time is 1 failure per 500 hours. Therefore, $\lambda = \dfrac{1}{500} = 0.002$, and the PDF is $f(x) = 0.002 e^{-0.002x}$ on the interval $[0,\infty)$.

The CDF of this function is $F_X (x) = 1 - e^{-0.002x}$. Therefore, the probability we seek is

\begin{equation} P(T > 600) = 1 - F(600) = 1 - (1 - e^{-0.002(600)}) = e^{-1.2} \approx 0.3012 \end{equation}

Derivation of the Formulas

Recall that the PDF of the Poisson distribution is $P(X=x) = \dfrac{\lambda^x e^{-\lambda}}{x!}$, where $X$ is the number of successes for some unit of time, and $\lambda$ is the mean number of successes per unit time. An immediate application of this formula is that the probability of zero successes in that unit of time is $P(X=0) = e^{-\lambda}$.

Now the Poisson distribution does require that events are independent of one another. Therefore, if we want to find the probability of zero successes in two units of time, we can do so by multiplying. That is:

\begin{align} P(X=0 \text{ in two time units}) &= P(X=0 \text{ in first time unit}) \cdot P(X=0 \text{ in second time unit}) \\ &= e^{-\lambda} \cdot e^{-\lambda} \\ &= e^{-2\lambda} \end{align}

This process can be extended to any number of time units. Therefore, the probability of zero successes in $t$ time units is

\begin{equation} P(X=0 \text{ in } t \text{ time units}) = e^{-t\lambda} = e^{-\lambda t} \end{equation}

But the probability of zero successes in $t$ time units is logically the same as the probability that the waiting time until the first success is greater than $t$ time units. In other words, if $T$ is the random variable for the time until the first success, we have

\begin{equation} P(T > t) = P(X=0 \text{ in } t \text{ time units}) = e^{-\lambda t} \end{equation}

By the complementary rule, we can produce the CDF for this waiting time distribution.

\begin{equation} F_T (t) = P(T \le t) = 1 - P(T > t) = 1 - e^{-\lambda t} \end{equation}

Differentiating will produce the PDF.

\begin{equation} f(t) = \dfrac{d}{dt} F_T (t) = \dfrac{d}{dt} (1 - e^{-\lambda t}) = \lambda e^{-\lambda t} \end{equation}

Changing the variable $t$ to $x$ will produce the PDF formula given above. The CDF given above can also be produced by changing the variable in the CDF formula of this derivation.

The moment generating function is obtained as the value of $E(e^{tX})$.

\begin{align} M(t) &= E(e^{tX}) = \int_0 ^\infty \lambda e^{tx} e^{-\lambda x} \mathrm{d}x \\ &= \int_0 ^\infty \lambda e^{t-\lambda x} \mathrm{d}x \\ &= \left. \dfrac{\lambda}{t-\lambda} e^{(t-\lambda)x} \right|_{x=0}^\infty \\ &= \dfrac{\lambda}{\lambda - t} \end{align}

We can derive the expected value, $E(X)$, from the first derivative of the moment generating function.

\begin{align} M'(t) &= \lambda (-1)(\lambda - t)^{-2} (-1) = \lambda (\lambda - t)^{-2} \\ E(X) &= M'(0) = \lambda \lambda^{-2} = \dfrac{1}{\lambda} \end{align}

The values of $E(X^2)$ and the variance can be obtained from the second derivative of the moment generating function.

\begin{align} M''(t) &= \lambda (-2) (\lambda - t)^{-3} (-1) = 2 \lambda (\lambda - t)^{-3} \\ E(X^2) &= M''(0) = 2 \lambda \lambda^{-3} = \dfrac{2}{\lambda^2} \\ Var(X) &= E(X^2) - (E(X))^2 = \dfrac{2}{\lambda^2} - \left( \dfrac{1}{\lambda} \right)^2 = \dfrac{1}{\lambda^2} \end{align}

And therefore, the standard deviation is $\sigma_X = \dfrac{1}{\lambda}$.