search

gkichaev / PAINTOR_V3.0

Fast, integrative fine mapping with functional data
55 stars 21 forks source link

Comparing -enumerate and -mcmc runs #37

Open ppericard opened 5 years ago

ppericard commented 5 years ago

Hi,

I'm running PAINTOR using several parameters combination on the same datasets and I'm trying to compare -enumerate and -mcmc runs.

When comparing the base (no annotation) runs I'm wondering if it makes sense to compare directly the log bayes factor obtained at the end of the 2 runs (enumerate 2 and mcmc). Because I end up with a huge difference that is not that intuitive for me. The -mcmc -burn_in 50000 -max_samples 10000 -num_chains 5 run, that took 2.5h of computing time, ends up with an average log bayes factor of 643.571, while the -enumerate 2 run took less than 4h and ended up with an average log bayes factor of 29813.6. Since the MCMC algorithm has to go through less space than the exact algorithm of enumerate, I would have expected the MCMC run to be either much faster, or end up with a similar score than the enumerate run.

Is the bayes factor an absolute metric of how good the result is for a given dataset (either with or without annotations) ? If not, what would be the best way to compare runs between them ?

Thanks in advance