Jeffreys’ prior: For the multivariate normal model, Jeffreys’ rule for generating a prior distribution on $(θ, Σ)$ gives $p_J(θ, Σ) ∝ |Σ|^{−(p+2)/2} .$
a) Explain why the function $p_J$ cannot actually be a probability density for $(θ, Σ)$.
$$ p_J(\theta, \Sigma) \propto |\Sigma|^{-(p+2)/2} $$
$\int \int |\sigma|^{-(p+2)/2} d\theta d\sigma = \int |\sigma|^{-(p+2)/2} \theta|^{\infty}_{-\infty} d\sigma$
Uniform 분포는 범위가 있어야 하지만 위 적분값은 $-\infty$에서 $\infty$ 사이의 범위를 가지므로 ($\theta, \Sigma$)의 probability density의 적분값이 1이 되지 않는다.
b) Let $p_J (θ, Σ|y_1, . . . , y_n)$ be the probability density that is proportional to $p_J (θ, Σ)×p(y_1, . . . , y_n|θ, Σ)$. Obtain the form of $p_J (θ, Σ|y_1, . . . , y_n)$, $p_J (θ|Σ, y_1 , . . . , y_n)$ and$p_J (Σ|y_1 , . . . , y_n).$
The inverse-Wishart($v_0, S^{-1}_0$) density :
$$ p(\Sigma)= \left[ [2^{v_0p/2} \pi^{{p \choose 2}/2} |S_0|^{-v_0/2} \prod_{j=1}^p \Gamma([v_0+1-j]/2] \right]^{-1} \\ \times |\Sigma|^{-(v_0+p+1)/2} \times exp \left\{ {-tr(S_0 \Sigma^{-1})/2} \right\}. $$
In second line, since $tr(A) = \sum_{j=1}^p a_{jj}$ for a square $p\times p$ matrix $A$.
The sampling distribution for $Y_1,...,Y_n$ is
$p(y_1,...,y_n| \theta, \Sigma) = (2\pi)^{-np/2} |\Sigma|^{-n/2}exp \left\{ -\sum_{i=1}(y_i-\theta)^T \Sigma^{-1} (y_i-\theta)/2 \right\}$
$p_J (θ, Σ|y_1, . . . , y_n) \propto p_J (θ, Σ)×p(y_1, . . . , y_n|θ, Σ)$
$\propto \left[ |\Sigma|^{-(p+2)/2 } \right] \times \left[ |\Sigma|^{-n/2} exp(-tr(S_\theta \Sigma^{-1})) \right]$
$\propto |\Sigma|^{-(p+n+2)/2} exp(-tr(S_\theta\Sigma^{-1})/2$
Parameter의 full conditional을 구하기 위해서 다른 파라미터들을 상수로 가정했을 때,
$p_J(\theta |y_1, ...,y_n, \Sigma) \propto exp(-tr(S_\theta \Sigma^{-1})/2)$