search

xavierdidelot / BactDating

Bayesian inference of ancestral dates on bacterial phylogenetic trees
https://xavierdidelot.github.io/BactDating
MIT License
80 stars 15 forks source link

Model Comparison #40

Closed dnanto closed 3 years ago

dnanto commented 3 years ago

I have a question regarding interpretation of results. I ran four models on the same data set and obtained the following table. The modelcompare function will say that the mixedgamma model outperforms the others, since it has the lowest dic value. However, the ESS values for this model are super low and I would say that the poisson model is the best. I realize that there is a lot of art involved in the interpretation, but I'm curious if there is a more algorithmic method for model selection, at least within this framework. I'm also curious and not sure why a model with low ESS would also have a low dic value. Also, is there a way to estimate the thinning parameter and number of iterations beforehand to guarantee a sufficient ESS value? Anyway, thank you, I've had a lot of success with this package and have results comparable to BEAST.

model mu sigma alpha dic
mixedgamma 13.06311837 8.551078275 111.3273616 113.672973
poisson 158.0832067 0 4075.570107 344.610655
relaxedgamma 12.27704162 6.988287349 138.6838716 114.551416
strictgamma 174.3190075 0 4343.889795 463.255654
xavierdidelot commented 3 years ago

Yes it seems that mixedgamma is the best model, but for the comparison of models to be meaningful it is necessary to have good ESS values first. Note that here mixedgamma is probably exactly the same as relaxedgamma here. So your table shows that the clock is relaxed, but fitting the relaxedgamma clock model is taking a long time. I would suggest trying the new arc model described in the paper below, which tends to give better and more stable results in less iterations.

Didelot, X., Siveroni, I., and Volz, E. M. 2020. Additive uncorrelated relaxed clock models for the dating of genomic epidemiology phylogenies. Molecular Biology and Evolution:10.1093/molbev/msaa193.

shaodongyan commented 1 year ago

Sorry,But I want to know a question. Why the poisson model's sigma is 0? I meet this problem too. I want to know If the possion model 's sigma is zero, how to choose the best model?

xavierdidelot commented 1 year ago

When you use the Poisson model, the output sigma will always be zero. This is not a problem but expected behaviour. It happens because the Poisson model does not have a sigma parameter, and so the value zero is returned for sigma, but it does not mean anything about whether the Poisson model is suitable or not.