mdietze
commented
8 years ago Need a bit more context, what analysis is the user asking about?
- For most of the basic stuff in the user interface (doing a single run, sensitivity/uncertainty analysis) a p-value isn't something meaningful to ask for because we're not comparing to anything.
- For the ensemble analysis, we are explicitly generating a credible interval
- For parameter data assimilation and the meta-analysis, we are explicitly generating posterior distributions. Admittedly, the summary table for those parameters with a SE and CI may be a bit buried -- that seems like something we could improve. However, to calculate a tail probability you would need an appropriate null hypothesis -- I'm not sure what the null hypothesis would be for parameters in process-based models or what the user would do with such information. For example, what would you learn if Vcmax wasn't significantly different from zero? Furthermore, since Vcmax can't be negative, it should have already had a prior that prohibited negative values, so the tail probability of being less than 0 would be 0 by definition.
- Similar argument exists for the state-assimilation. e.g. you can't test the probability that GPP < 0 if the model can't predict GPP < 0
- For benchmarking, we will be calculating any metric that is selected. If someone wants to create a p-value metric function, I won't stop them on philisophical ground (i.e. I won't be dogmatically Bayesian). I would stop them if the metric was inappropriate (e.g. treating the deterministic temporal variability in model output as model error -- just because you can calculate a standard error for something, doesn't make it the right SE to put in a t-test)
Refraining from taking the bait about p-values not proving something right, or that there will never be p-values in a Bayesian analysis.
github-actions[bot]
A user asks