probml / pml2-book

Probabilistic Machine Learning: Advanced Topics
MIT License
1.39k stars 120 forks source link

v2024-06-26: Typo in 3.8.3.2 "Harmonic Mean Estimator" (p. 121/122) #339

Open BlackHC opened 2 months ago

BlackHC commented 2 months ago

The text currently says that the samples are from the prior. However, the samples need to be from the posterior as the derivation in eq (3.284) ff shows.

So, the required changes are:

  1. $\theta_s \sim p(\theta \mid \mathcal{D})$ are samples from the posterior.
  2. Strike "(We have assumed the prior is proper, so it integrates to 1.)"
  3. Rewrite the following sentence by citing Radford Neal's blog post more directly maybe:

    It’s easy to see why this estimator can’t possibly work well in practice. As is well-known, the posterior distribution for a Bayesian model is often much narrower than the prior, and it is often not very sensitive to what the prior is, as long as the prior is broad enough to encompass the region with high likelihood. [...] Since the prior affects the harmonic mean estimate of the marginal likelihood only through its effect on the posterior distribution, it follows that the harmonic mean estimate is very likely to be virtually the same for the two priors, for any reasonable size sample from the posterior. [...] The harmonic mean method is clearly hopelessly inaccurate.

Here is the relevant excerpt from the current book draft/PDF:

image