The normal model

A random variable $Y$ is said to be normally distributed with mean $θ$ and variance $σ^2 > 0$ if the density of $Y$ is given by

$$ p(y|θ, σ^2) = \frac{1}{\sqrt{2πσ^2}}e^{−\frac{1}{2}(\frac{y−θ}{σ})^2}, −∞ < y < ∞ $$

• the distribution is symmetric about $θ$, and the mode, median and mean are all equal to $θ$;
• about 95% of the population lies within two standard deviations of the mean (more precisely, 1.96 standard deviations);

Inference for the mean, conditional on the variance

Suppose our model is {$Y_1, . . . , Y_n|θ, σ^2$} $∼ i.i.d.$ normal $(θ, σ2)$. Then the joint sampling density is given by

$p(y_1, . . . , y_n|θ, σ^2) = \prod_{i=1}^n=p(y_i|θ, σ^2)$

                            $= \prod_{i=1}^n \frac{1}{\sqrt{2πσ^2}}e^{−\frac{1}{2}(\frac{y−θ}{σ})^2}$

                            $= (2\pi\sigma^2)^{-n/2}exp \begin{Bmatrix}-\frac{1}{2}\sum (\frac{y_i-\theta}{\sigma})^2 \end{Bmatrix}$

Inference for this two-parameter model can be broken down into two one-parameter problems. We will begin with the problem of making inference for $θ$ when $σ^2$ is known, and use a conjugate prior distribution for $θ$. For any (conditional) prior distribution $p(θ|σ^2)$, the posterior distribution will satisfy

$p(\theta|y_1,...,y_n,\sigma^2) \propto p(θ|σ^2) × e^{−\frac{1}{2σ^2}\sum(yi−θ)^2}$

                            $\propto p(\theta|\sigma^2) \times e^{c_1(\theta-c_2)^2}$

The simplest such class of probability densities on $\mathbb{R}$ is the normal family of densities, suggesting that if $p(θ|σ^2)$ is normal and $y_1, . . . , y_n$ are $i.i.d.$ normal$(θ, σ^2)$, then $p(θ|y_1, . . . , y_n, σ^2)$ is also a normal density. Let’s evaluate this claim: If $θ ∼$ normal $(µ_0, τ^2_0)$, then

$$ p(θ|y1, . . . , yn, σ2) = p(θ|σ2)p(y1, . . . , yn|θ, σ2)/p(y1, . . . , yn|σ2) $$

                                                 $∝ p(θ|σ^2)p(y_1, . . . , y_n|θ, σ^2)$

                                                 $∝ exp \left\{− \frac{1}{2τ^2_0}(θ − µ_0)^2 \right\} exp \left\{−\frac{ 1}{2σ^2}\sum(y_i − θ)^2\right\}.$