Distributions of Random Variables

1 Normal Distribution

The normal distribution (Gaussian distribution) is a symmetric, bell-shaped distribution:

The exact form of the normal pdf is:

\[\text{pdf}(X) = \frac{1}{\sqrt{2\pi}\sigma} e^{-(X-\mu)^2/2\sigma^2}\]

Note in this expression there are only really two variables, \(\mu\) and \(\sigma\). These are the only parameters required to specify a normal curve (everything else in the expression is a constant). Changing \(\mu\) shifts the curve left/right, and increasing \(\sigma\) makes the curve wider.

If \(X\) a normally distributed RV with mean \(\mu\) and variance \(\sigma^2\), you can use the shorthand notation:

\[X \sim \mathcal N(\mu, \sigma^2)\]

The distribution in the figure above is a specific case called the standard normal distribution. It has \(\mu = 0\) and \(\sigma = 1\).

You can find probabilities under the standard normal curve in R using the pnorm() function. It gives the cumulative probability that \(X \leq k\).

E.g. to compute \(\text{P}(X \leq 0)\):

pnorm(0)
## [1] 0.5

To compute \(\text{P}(-1 \leq X \leq 1)\):

pnorm(1) - pnorm(-1)
## [1] 0.6826895

To compute \(\text{P}(-2 \leq X \leq 2)\):

pnorm(2) - pnorm(-2)
## [1] 0.9544997

And \(\text{P}(-3 \leq X \leq 3)\):

pnorm(3) - pnorm(-3)
## [1] 0.9973002

These probabilities are characteristic properties of the normal distribution. ~68% of values lie within 1 standard deviation of the mean, ~95% of values lie within 2 standard deviations of the mean, and ~99.5% of values lie within 3 standard deviations of the mean.

Consider a normally distributed random variable \(X\) with \(\mu = 10\) and \(\sigma = 2\). What is the probability that \(X\) is smaller than 7? One way to find this is by determining the \(Z\)-score.

The \(Z\)-score of an observation is defined as:

\[Z = \frac{X - \mu}{\sigma}\]

The \(Z\)-score describes how far away the value is from the mean, in units of standard deviation. In this case, with the observed value \(X = 7\), the \(Z\)-score is \((7 - 10) / 2 = -1.5\). This means the observed value is 1.5 standard deviations to the left of the mean.

The \(Z\)-score is essentially a mapping of \(X\) onto the standard normal distribution. Thus

\[\text{P}(X \leq 7) = \text{P}(Z \leq -1.5)\]

So to compute \(\text{P}(X \leq 7)\) for \(X \sim \mathcal N(10, 2^2)\), you can use pnorm(), plugging in the \(Z\)-score:

pnorm(-1.5)
## [1] 0.0668072

Alternatively you can explicitly specify the parameters of normal distribution as additional arguments:

pnorm(7, mean = 10, sd = 2)
## [1] 0.0668072

2 Uniform Distribution

The uniform distribution (rectangular distribution) can model a random variable which has an equal probability to take any value in a specified interval.

The uniform distribution is specified by two parameters, \(a\) and \(b\), which define the lower and upper bounds of the interval. The probability distribution is:

\[X \sim \mathcal U(a,b) \hspace{0.5cm} \longrightarrow \hspace{0.5cm} \text{pdf}(X) = \begin{cases} \frac{1}{b-a} & \text{for}\;\; x \in [a,b] \\ 0 & \text{otherwise} \end{cases}\]

The uniform distribution can be either discrete or continuous. In the discrete case, \(X\) can take any integer value between \(a\) and \(b\) with equal probability, and in the continuous case, \(X\) can take any real value between \(a\) and \(b\) with equal probability.

Expected value of a uniform distribution:

\[\text{E}[X] = \frac 12 (a+b)\]

Variance of a uniform distribution:

\[\text{Var}[X] = \frac{(b-a)^2-1}{12} \hspace{0.5cm} \text{(discrete)}\]

\[\text{Var}[X] = \frac{(b-a)^2}{12} \hspace{0.5cm} \text{(continuous)}\]

3 Bernoulli Distribution

The Bernoulli distribution can model a discrete random variable with a binary outcome (e.g. a single coin toss). Bernoulli random variables have the following features:

The Bernoulli distribution is specified by one parameter, \(p\), which describes the probability of a success (\(X=1\)).

The probability distribution of a Bernoulli RV is:

\[X \sim \text{Ber}(p) \hspace{0.5cm} \longrightarrow \hspace{0.5cm} \text{pmf}(X) = \begin{cases} p & X=1 \;\; \text{(success)} \\ 1-p & X=0 \;\; \text{(failure)}\end{cases}\]

Expected value of a Bernoulli RV:

\[\text{E}[X] = p\]

Variance of a Bernoulli RV:

\[\text{Var}[X] = p(1-p)\]

4 Binomial Distribution

The binomial distribution can model multiple occurrences of a Bernoulli process (e.g. several coin tosses). Binomial random variables have the following features:

The binomial distribution is specified by two parameters, \(n\) (number of trials) and \(p\) (probability of success), and its probability distribution is:

\[X \sim \text{Bin}(n,p) \hspace{0.5cm} \longrightarrow \hspace{0.5cm} \text{P}(X = k) = \begin{pmatrix} n \\ k\end{pmatrix} p^k (1-p)^{n-k}\]

where \(\big( \begin{smallmatrix} n \\ k \end{smallmatrix} \big)\) is called the binomial coefficient, and is another way of writing \(\frac{n!}{k!(n-k)!}\).

Expected value of a binomial RV:

\[\text{E}[X] = np\]

Variance of a binomial RV:

\[\text{Var}[X] = np(1-p)\]

Note the Bernoulli distribution is a special case of the binomial with \(n = 1\).

5 Poisson Distribution

The Poisson distribution can model a discrete random variable for events that occur a fixed number of times during a given interval. Poisson random variables have the following features:

The Poisson distribution is specified by one parameter, \(\lambda\), known as the event rate, which describes the average number of events in a given interval. The probability distribution is:

\[X \sim \text{Pois}(\lambda) \hspace{0.5cm} \longrightarrow \hspace{0.5cm} P(X = k) = \frac{e^{-\lambda} \lambda^k}{k!}\]

Expected value of a Poisson RV:

\[\text{E}[X] = \lambda\]

Variance of a Poisson RV:

\[\text{Var}[X] = \lambda\]