AdityaSavara
commented
4 years ago I have thought about how to do this. It seemed to me that one could not use the HPD integrations because those would not be directly scalable to each other. The idea I came up with was to integrate across a fixed number of standard deviations of the input parameters. After inspecting the definition of Bayes Factor more closely, that does seem to be consistent with my idea.
It's already on the to-do list to get the posterior from a grid search. If we do that correctly, we should be able to use it to scan across parameter space for each model (up to a certain number of standard deviations out) and to get Bayes factor. https://en.wikipedia.org/wiki/Bayes_factor
Another alternative is to use a different backend like Zeus or pymc3 for model discrimination, since it's possible they will have something built-in.
https://colcarroll.github.io/pymc3/notebooks/model_comparison.html https://docs.pymc.io/notebooks/GLM-model-selection.html https://github.com/pymc-devs/pymc3/issues/938
Model Discrimination / Model Comparison / Model Selection is currently not implemented. If anybody needs it, direct them to this issues card.
The current plan is as follows: if anybody needs it, and if Eric is available, he will implement it and guide the usage, then become a middle author. If Eric needs to consult Ashi, then Ashi can be put in the acknowledgements.
When first implementing the feature, to make sure it works, the snail example might be something Eric can use.
Ashi notes:
I think that the syntax from the user’s point of view will have to be something like this.
UserInput1 …. CKPQ.PE_Object1(UserInput1)
UserInput2 …. CKPQ.PE_Object2(UserInput2)
ModelRelativeWeights = [0.20, 0.80] #We can normalize this to 1. So they could even put in 200 and 800 for example.
CKPQ.ModelComparison(PE_Object1,PE_Object2)
Eric will need to implement because I am not sure how to do it. I have done it for some simple “on paper” cases, but here we are using sampling and MCMC with probability density functions, so I’m not sure how to do it here.