joshspeagle / dynesty

Dynamic Nested Sampling package for computing Bayesian posteriors and evidences
https://dynesty.readthedocs.io/
MIT License
357 stars 77 forks source link

Usage of KLD for stopping criteria #331

Closed segasai closed 3 years ago

segasai commented 3 years ago

I've been looking at the stopping criteria for the dynamic sampling and I kind'a struggle to understand the rationale why the mean(KLD)/stddev(KLD) for KLD that's always positive is a meaningful statistic to optimize when deciding when to stop.

If I understand correctly the most likely thing people will care about for the posterior is the effective number of samples, as I think that'll dominate the error-budget, as opposed to the volume misestimates (which is what the default stopping function with jitter_run tries to estimate)

Also in fact the 'weight_function' tries to essentially find the best logl range to maximize Neff (minimizing the maximum weights) , so therefor it's kind'a weird that the stop function is measuring something else.

joshspeagle commented 3 years ago

I'm perfectly happy switching over the default choice of stopping criteria to some combination of Neff and logzerr instead of the current KLD_SNR and logzerr weighting. I believe that functionality was added to the code a while back, so it's an easy change to make. The default number of samples can probably be chosen so that the mean and/or covariance error can be bounded to below some threshold (assuming a Gaussian distribution).