search

BEAST2-Dev / nested-sampling

Nested sampling packagin for BEAST
GNU Lesser General Public License v2.1
11 stars 1 forks source link

Marginal likelihood output #8

Closed ksw9 closed 3 years ago

ksw9 commented 4 years ago

Dear Remco,

Thank you for another really helpful tool!

I'm hoping I can confirm that I am interpreting the output of NS correctly. I'm comparing BEAST runs with different clock and demographic models. Is the Marginal likelihood reported in the nested sampling output the log marginal likelihood? So, for instance, if I have two different models, M1 and M2, then I could just subtract the log marginal likelihoods to get the Bayes factor?

M1: strict clock, constant population Marginal likelihood: -5901035.1366844885 (bootstrap SD=25.790381165909494)

M2: strict cloc, skyline population size Marginal likelihood: -5900976.683923649 (bootstrap SD=22.68462670702141)

BF = -5901035.1366844885 - -5900976.683923649 = -58

Would the correct interpretation be that there is significant support for M2?

Thank you again for helping with the interpretation!

All the best,

rbouckaert commented 3 years ago

@ksw9 you are correct: it is the log-marginal likelihood, not the actual marginal likelihood, which is a number between 0 and 1. You can tell from the negative sign in front of the number that it cannot be a probability and the log-form is used, and out of lazynes, the "log" part is omitted. To make things confusing, the SD is the actual standard deviation of the log-marginal likelihood.

The log Bayes factor is the difference between the two log-marginal likelihoods, which is about 58 in favour of M2. However, the SDs are about 26 and 23 in the estimates of the log marginal likelihoods, so this BF can be due to chance. Running it again with larger numbers of points can reduce the uncertainty in the estimates (see FAQ under How many particles do I need?).

ksw9 commented 3 years ago

Dear Remco, Thank you so much for the explanation – this helps a lot. And thanks for all your work on BEAST. Best,

From: Remco Bouckaert notifications@github.com Date: Sunday, October 18, 2020 at 1:10 PM To: BEAST2-Dev/nested-sampling nested-sampling@noreply.github.com Cc: Katharine Walter kwalter@stanford.edu, Mention mention@noreply.github.com Subject: Re: [BEAST2-Dev/nested-sampling] Marginal likelihood output (#8)

@ksw9https://github.com/ksw9 you are correct: it is the log-marginal likelihood, not the actual marginal likelihood, which is a number between 0 and 1. You can tell from the negative sign in front of the number that it cannot be a probability and the log-form is used, and out of lazynes, the "log" part is omitted. To make things confusing, the SD is the actual standard deviation of the log-marginal likelihood.

The log Bayes factor is the difference between the two log-marginal likelihoods, which is about 58 in favour of M2. However, the SDs are about 26 and 23 in the estimates of the log marginal likelihoods, so this BF can be due to chance. Running it again with larger numbers of points can reduce the uncertainty in the estimates (see FAQhttps://github.com/BEAST2-Dev/nested-sampling/wiki/FAQ under How many particles do I need?).

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/BEAST2-Dev/nested-sampling/issues/8#issuecomment-711415317, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ABYOQ74AFMSZIWNSDTBSTUDSLNDS3ANCNFSM4QNPQGIQ.