search

simetenn / uncertainpy

Uncertainpy: a Python toolbox for uncertainty quantification and sensitivity analysis, tailored towards computational neuroscience.
http://uncertainpy.readthedocs.io
GNU General Public License v3.0
220 stars 50 forks source link

use bayesian calibration and then apply uncertainpy to do a GSA? #50

Closed lilianschuster closed 3 years ago

lilianschuster commented 3 years ago

Dear Uncertainpy developers,

I am calibrating a glacier model with a Bayesian inference model, i.e. I assume prior distributions for two free parameters and incorporate uncertainties of the observations that are used for calibration. So, after calibration I get posterior distributions of the two parameters and I can use those to estimate uncertainties of the model output. I would like to assess the influence of the individual parameters on the total uncertainty. So, I thought that a Global Sensitivity Analysis after the Bayesian calibration (on the posterior distribution) could give me some quantitative estimates (more detailed infos are here) . Did you try out something similar to that by chance? I was thinking about combining the Bayesian calibration (using the PyMC3 package) with uncertainpy? Would it be a possible way that I use the posterior distributions of the parameters from the Bayesian calibration as input parameters (e.g. just by prescribing mean and std) and then do the sensitivity analysis with uncertainpy?

If you don't know if this can be done / is possible, no problem. I just thought that you might know an answer to that.

Greetings, Lilian

simetenn commented 3 years ago

Hi Lilian,

Unfortunately, I have never tried something like that, but If it is what I think it is I have thought of it as something that would be interesting to do.

I do not have much experience with Bayesian parameter estimation (which it sounds like you do for your two parameters?), so please correct me if/where I am wrong. To summarize your problem (for my sake), you have a model with two parameters that you fit to your data, and you get the distributions for the two parameters that best fit the observations you have, and which includes the uncertainty in your observations?

If you have the distributions of your parameters it should in theory be possible to use those distributions in Uncertainpy to calculate the uncertainty of your model. How are your distributions defined? Is it just a normal distribution since you mention mean and standard deviation? Uncertainpy requires that you define them as Chaospy distributions, which perhaps might be difficult depending on how you get the distributions from pymc3. But Chaospy has support for a lot of different distributions.

lilianschuster commented 3 years ago

Hi, Thanks a lot for your answer! It is already very helpful to know that you have not tried something like that. I think you understood me right. The posterior distributions of the two parameters from the Bayesian parameter calibration include the uncertainties in the observations (and those from equifinality). These distributions can be approximated to be gaussian. However, these two parameters, hence their distributions, are correlated, so the input chaospy distribution would need to be two-dimensional. I am not sure if I can prescribe a 2D-distribution in chaospy but I will try it out at some point.

simetenn commented 3 years ago

That is no problem with Chaospy: https://chaospy.readthedocs.io/en/master/user_guide/fundamentals/probability_distributions.html#Multivariate-variables

And if you need dependent variables: https://chaospy.readthedocs.io/en/master/user_guide/advanced_topics/stochastic_dependencies.html