Stefan Siegert

The multivariate Normal distribution in canonical form

\( \def\fat#1{\boldsymbol{#1}} \)

Let \(\fat{x}\) be a random vector with multivariate Normal distribution with expectation \(\fat{\mu}\) and variance matrix \(\Sigma\) \[\begin{aligned} & \fat{x} \sim \mathcal{N}(\fat{\mu}, \Sigma) \end{aligned}\] The pdf of \(\fat{x}\) is given by \[\begin{aligned} p(\fat{x}) = |2\pi\Sigma|^{-1/2} \exp-\frac{1}{2}\left\{ (\fat{x} - \fat{\mu})'\Sigma^{-1}(\fat{x} - \fat{\mu}) \right\}& \end{aligned}\]

Defining the precision matrix \(Q=\Sigma^{-1}\), the log pdf of \(\fat{x}\) can be written as \[\begin{aligned} \log p(\fat{x}) & = const - \frac{1}{2} (\fat{x} - \fat\mu)'Q(\fat{x}-\fat\mu)\\ & = const' - \frac{1}{2} \fat{x}'Q\fat{x} + (Q\fat{\mu})'\fat{x} \end{aligned}\] where the additive constants do not depend on \(\fat{x}\).

That is whenever we have a random vector \(\fat{x}\) whose log-pdf can be written in the form \[\begin{aligned} & \log p(\fat{x}) = const - \frac{1}{2} \fat{x}'Q\fat{x} + \fat{b}' \fat{x}, \end{aligned}\] for some constant (positive-definite) matrix \(Q\), and some constant vector \(\fat{b}\), then we know that \(\fat{x}\) has a multivariate Normal distribution with expectation \(Q^{-1}\fat{b}\) and variance matrix \(Q^{-1}\) \[\begin{aligned} & \fat{x} \sim \mathcal{N}(Q^{-1}\fat{b}, Q^{-1}). \end{aligned}\]

The Normal distribution written in terms of \(Q\) and \(\fat{b}\) is said to be in canonical form. For a Normal random vector with density given in canonical form with parametrs \(Q\) and \(\fat{b}\) we write \[\begin{aligned} & \fat{x} \sim \mathcal{N}_c(\fat{b}, Q). \end{aligned}\]