Tag Archives: Covariance

Variance and Expected Value Algebra

Linearity of Expected Value: Suppose X and Y are random variables and a and b are scalars.  The following relationships hold:

E[aX+b] = aE[X]+b

E[aX+bY] = aE[X] +bE[Y]


Var(aX+bY) = a^2Var(X)+2abCov(X,Y)+b^2Var(Y)

Suppose X_i for i=\left\{1\ldots n\right\} are n independent identically distributed (iid) random variables.  Then Cov(X_i,X_j) = 0 for i\ne j and

\displaystyle Var\left({\sum_{i=1}^n X_i}\right) = \sum_{i=1}^n Var(X_i)


X is the stock price of AAPL at market close.  Y is the sum of closing AAPL stock prices for 5 days.  Then

\begin{array}{rl} Var(Y) &= \displaystyle \sum_{i=1}^5 Var(X_i) \\ &= 5Var(X) \end{array}.  

Contrast this with the variance of Z = 5X.  In other words, Z is a random variable that takes a value of 5 times the price of AAPL at the close of any given day.  Then

\begin{array}{rl} Var(Z) &= Var(5X) \\ &=5^2Var(x) \end{array} 

The distinction between Y and Z is subtle but very important.

Variance of a Sample Mean:

In situations where the sample mean \bar{X} is a random variable over n iid observations (i.e. the average price of AAPL over 5 days), the following formula applies:

\begin{array}{rl} Var(\bar{X}) &= \displaystyle Var\left(\frac{1}{n} \displaystyle \sum_{i=1}^n X_i\right) \\ &= \displaystyle \frac{nVar(X)}{n^2} \\ &= \displaystyle \frac{Var(X)}{n} \end{array} 


Filed under Probability

Functions and Moments

Some distribution functions:

Survival function

\displaystyle S(x) = 1-F(x) = \Pr(X>x)  

where F(x) is a cumulative distribution function.

Hazard rate function

\displaystyle h(x) = \frac{f(x)}{S(x)} = -\frac{d\ln{S(x)}}{dx}

where f(x) is a probability density function.

Cumulative hazard rate function

\displaystyle H(x) =\int_{-\infty}^x{h(t)dt} = -\ln{S(x)}

The following relationship is often useful:

S(x) = \displaystyle e^{-\int_{-\infty}^x{h(t)dt}}

Expected Value:

\displaystyle E[X] = \int_{-\infty}^\infty{xf(x)dx}

Or more generally,

\displaystyle E[g(X)] = \int_{-\infty}^\infty{g(x)f(x)dx}

When g(X) = X^n, the expected value of such a function is called the nth raw moment and is denoted by \mu'_n.  Let \mu be the first raw moment.  That is, \mu = E[X].  E[(X-\mu)^n] is called an nth central moment.

Moments are used to generate some statistical measures.

Variance \sigma^2

\displaystyle Var(X) = E[(X-\mu)^2] = E(X^2) - E(X)^2

The coefficient of variation is \displaystyle \frac{\mu}{\sigma}.

Skewness \gamma_1

\displaystyle \gamma_1 = \frac{\mu_3}{\sigma^3}

Kurtosis \gamma_2

\displaystyle \gamma_2 = \frac{\mu_4}{\sigma^4}

Covariance of two distribution functions

\displaystyle Cov(X,Y) = E[(X-\mu_x)(Y-\mu_Y)] = E[XY] - E[X]E[Y]

*Note: if X and Y are independent, Cov(X,Y)=0

Correlation coefficient \rho_{XY}

\displaystyle \rho_{XY} = \frac{Cov(X,Y)}{\sigma_X\sigma_Y}

All of the above definitions should be memorized.  Some things that might be tested in the exam are:

  • Given a particular distribution function, what happens to skewness or kurtosis in the limit of a certain parameter?
  • What is the expected value, variance, skewness, kurtosis of a given distribution function?
  • What is the covariance or correlation coefficient of two distribution functions?

Additional Notes

Central moments can be calculated using raw moments.  Know how to calculate raw moments using the statistics function on the calculator.  This can be a useful timesaver in the exam.  Using alternating positive and negative binomial coefficients, write an expression for \mu_n with \mu' and \mu as the two binomial terms.


\mu_4 = \mu'_4 - 4\mu'_3\mu + 6\mu'_2\mu^2 - 4\mu'_1\mu^3 + \mu^4

Since \mu'_1 = \mu, the two terms on the end simplify to -3\mu^4.  The result is

\mu_4 = \mu'_4 - 4\mu'_3\mu + 6\mu'_2\mu^2 - 3\mu^4

Moment Generating Function:

If the moment generating function M(t) is known for random variable X, it’s nth raw moment can be found by taking the nth derivative of M(t) and evaluating at 0.  Moment generating functions take the form:

M(t) = \displaystyle E[e^{tx}]

If Z = X +Y, then M_Z(t) = M_X(t)\cdot M_Y(t).

Leave a comment

Filed under Probability