ja-vazquez / SimpleMC

Updated version of a simple MCMC code for cosmological parameter estimation where only expansion history matters.
GNU General Public License v2.0
21 stars 14 forks source link

MCEvidence and Maximum Likelihood Estimations #37

Closed camarman closed 1 year ago

camarman commented 2 years ago

I have run LCDM model with HD dataset (via default options) by using mcmc, which gives

SUMMARY
-------
analyzer: mcmc
mc_evidence: 
log-Evidence with mcevidence: [-18.38013554 -18.38407484]
Burn-in fraction: 0e+00

maxlike: 7.2593
gr_diagnostic: None
Om: 0.3455 +/- 0.0747
Obh2: 0.0220 +/- 0.0005
h: 0.6699 +/- 0.0473

Elapsed time: 0.950 minutes = 57.018 seconds 
951 minutes = 57.040 seconds 

and later on with nested and the results are

SUMMARY
-------
analyzer: nested
nested_algorithm: multi
dynamic: False
maxlike: 7.4502
nlive: 50
niter: 175
ncall: 1683
%eff: 13.4284
logz: -12.7311 +/- 0.6000
Om: 0.3363 +/- 0.0697
Obh2: 0.0221 +/- 0.0004
h: 0.6709 +/- 0.0439

Elapsed time: 0.130 minutes = 7.825 seconds 

As I am analyzing the datasets, I need to obtain $-2\ln \mathcal{L}_{max}$ and $\ln \mathcal{Z}$.

As to my knowledge, log-Evidence with mcevidence is corresponding to $\ln \mathcal{Z}$. However, if that also corresponds to logz in nested-summary, then there seems to be a discrepancy ?

In both cases maxlike matches. Does maxlike corresponds to $-\ln \mathcal{L}_{max}$ ? Or does it corresponds to something else ?

In summary, my question is, what is the equations/variables that corresponds to maxlike, logz and log-Evidence.

camarman commented 1 year ago

Okay so I have found out that maxlike is $-\mathcal{L}_{\max}$ and logz is the $\ln \mathcal{Z}$ in dynesty. log-Evidence is is also $\ln \mathcal{Z}$ but the calculations are done over MCEvidence program. I guess they are different because MCMC and nested samplers have different methods and MCMC cannot constrain well $\ln \mathcal{Z}$

igomezv commented 1 year ago

Exactly. The maxlike value is just the maximum value of the likelihood during sampling, and since we are performing Bayesian inference, and not optimization, it is not the most relevant value (the sampling mean is more important), but it can sometimes be informative.

Regarding Bayesian evidence, two things should be kept in mind. First, as you mentioned, the sampling methods, nested sampling and Metropolis-Hastings have different ways of generating a sample. Secondly, the calculation of Bayesian evidence, nested sampling calculates Bayesian evidence at each sampling iteration as a Riemann sum (approximating the integral) and, on the other hand, MCEvidence approximates the Bayesian evidence value (roughly speaking) by clustering with the samples and relating to the density of points per volume. I believe that nested sampling is more robust because it directly calculates the integral and even estimates the uncertainty of the Bayesian evidence.

camarman commented 1 year ago

Thanks for the detailed answer. As you have pointed out, I have also realized that nested sampling is a much better method in terms of evaluating Bayesian evidence and comparing models.

Thanks for the help again. I guess I can close this issue since it's clear to me which equation corresponds to what.