Useful identities of multivariate Normal distribution

Normal distribution as exponential family

Normal distribution belongs in exponential family, so in its log PDF the 2 terms involving \(x\) has fixed form, which can be useful for deriving various identities.

For multivariate normal distribution:

  1. \(\mu^{T}\Sigma^{-1}x\)
  2. \(-\frac{1}{2}x^{T}\Sigma^{-1}x\)

Linear combination

Let weight distribution be \(N(\theta|\mu,\Sigma)\). Then

\[\begin{align} A\theta\sim N(A\mu,A\Sigma A^{T}) \end{align} \]

Posterior distribution

Let prior be \(N(\theta|\mu,\Sigma)\), likelihood be \(N(y|x^{T}\theta,\Sigma_{s})\). Then the posterior of \(\theta\) can be written as

\[\begin{equation} \theta|x \sim N((\Sigma^{-1}+x\Sigma_{s}^{-1}y)^{-1}(\Sigma^{-1}\mu+x\Sigma_{s}^{-1}y), \Sigma^{-1}+x\Sigma_{s}^{-1}y) \end{equation} \]

Predictive distribution

Let weight distribution be \(N(\theta|\mu,\Sigma)\), likelihood be \(N(y|x^{T}\theta,\Sigma_{s})\). Then the predictive distribution of \(x\) can be written as

\[\begin{equation} x|\theta \sim N(x^{T}\mu,x^{T}\Sigma x+\Sigma_{s}) \end{equation} \]

Conditional distribution

Let distribution be \(N(x|\mu,\Sigma)\). Split \(x\) into two half \(x_{1}\) and \(x_{2}\), and split \(\mu\) and \(\Sigma\) accordingly. (The first row of \(\Sigma\) contains \(\Sigma_{11}\) and \(\Sigma_{12}\).) The conditional distribution of \(x_{2}\) given \(x_{1}\) with values \(v_{1}\) can be written as

\[\begin{align} x_{2}|(x_{1}=v_{1}) &\sim N(\mu_{2|1},\Sigma_{2|1})\\ \mu_{2|1} &= \mu_{2} + \Sigma_{21}\Sigma_{22}^{-1}(v_{1}-\mu_{1})\\ \Sigma_{2|1} &= \Sigma_{2} - \Sigma_{21}\Sigma_{11}^{-1}\Sigma_{12} \end{align} \]