astro-informatics / harmonic

Machine learning assisted marginal likelihood (Bayesian evidence) estimation for Bayesian model selection
https://astro-informatics.github.io/harmonic/
GNU General Public License v3.0
54 stars 6 forks source link

How to read in 2D posterior samples (ndim x nsamples) #295

Closed sahiljhawar closed 4 months ago

sahiljhawar commented 4 months ago

I have posterior samples which is of the form ndim x nsamples and does not have chains. How can I parse these posteriors using harmonic? These posteriors are from MultiNest. I know MultiNest by default give Bayesian Evidence, however I want to see how robust harmonic is and how faithful it can reproduce the results.

jasonmcewen commented 4 months ago

Thanks for your question @sahiljhawar.

Posterior samples can be read in with the Chains object. However, you need posterior samples. Note that nested sampling techniques, like MultiNest, do not return posterior samples but samples from the constrained prior. You can nevertheless use these samples to compute posterior inference but you need to importance weight them, as usually consider for nested sampling. These weights could in principle be included in the harmonic estimator but they haven't been added as of yet since there is no real need to support nested sampling approaches as the evidence can already be calculated.

I would also caution against using MultiNest as the ground truth. If used carefully MultiNest can generally provide accurate evidence estimates, however there are a few recent papers highlight situations where MultiNest is not accurate (https://arxiv.org/pdf/2202.08233; https://arxiv.org/pdf/2404.16928).

A better comparison is probably just to run another sampler on your problem, such as emcee, and then using harmonic to compute the evidence from the corresponding posterior samples.

sahiljhawar commented 4 months ago

Thank you the detailed reply. I wasn't aware about the constrained prior, etc. I will read more about it. And thanks for the references as well.