Marginal Posterior for Bayes Factor

[pkroenert]

Hi there, first of all, this is an awesome package! I use it myself for regression problems.

As part of a regression analysis, I would like to compare two of my models. As far as I know, one way to do this in Bayesian statistics is to calculate the Bayes factor, for wich one needs the marginal posterior of both models.

How can I calculate the marginal posterior with Turing?

[pkroenert]

Is it just as simple as summing up the 'log_density' values wich can be quarried from the 'chain'?

[yebai]

@PToTheK Unfortunately summing up the log_density won't give you the correct marginal likelihood (aka model evidence). We have started adding algorithms for estimating marginal likelihood like AIS #1380 and Nested sampling https://github.com/TuringLang/Turing.jl/pull/1333. This PRs will likely take some more time before getting merged. In the meantime, you can take a look at the excellent loo package from the Stan community:

https://mc-stan.org/users/interfaces/loo

@torfjelde has recently written some docs on how to compute loo for Turing models.

https://arviz-devs.github.io/ArviZ.jl/dev/quickstart/#Additional-information-in-Turing.jl

[pkroenert]

@yebai Thank you for the answer!

However, summing up log_density should give me an estimate of the marginal posterior. Right?

[trappmartin]

Yes, it gives an estimate of the model evidence.

[trappmartin]

There is also a very interesting recent article linking the model evidence to the „leave-p-out cross-validation averaged over all p and all p-sized test sets“.

See: https://xianblog.wordpress.com/2020/10/09/marginal-likelihood-as-exhaustive-x-validation/

[pkroenert]

Thank's for the nice discussion!