Credible intervals and misspecification

[pawel-czyz]

Create the following experiment:

1. Sample $w \sim \mathrm{Uniform}(0.05, 0.95)$ many times, e.g., $S=200$.
2. For each weight $w$ construct a data set for some $N'=N=1000$ from a mixture of two Student distributions. One with $\nu = 3$, one with $mu_1 = 0$ and another at $\mu_2 = \delta$ for $\delta \sim \mathrm{Uniform}(0.5, 3)$. Dispersion should be of order 0.5.
3. Fit using HMC the following models:
   • Well-specified Student mixture.
   • Misspecified Gaussian mixture.
   • Partition the real axis into $K$ bins for different $K$ and fit the usual model.
4. Basing on the samples calculate the HDI credible intervals changing coverage.
5. Compare the credible interval coverage with the frequentist coverage. We expect that well-specified model will have approximately correct coverage, misspecified Gaussian mixture will have too small coverage, and binned model will have too large coverage (will be a bit more conservative than a correctly specified model).

For the figure:

1. Plot some data distribution.
2. Plot the posteriors (two panels).
3. Plot the coverage.

[pawel-czyz]

Resolved by #23