search

bbolker / discretization

opinion piece on discretization and multimodel averaging in ecological statistics
26 stars 4 forks source link

Smug Bayesians #2

Open oharar opened 5 years ago

oharar commented 5 years ago

You mention the problems with estimating CIs in regularised models. Of course, we Bayesians can be smug about this, and point to MCMC. An exponential prior is equivalent to a LASSO, but we can use other priors too. The PC priors approach (https://arxiv.org/abs/1403.4630) is an extension of this idea.

BertvanderVeen commented 2 years ago

It is straightforward to show with simulations that there are cases that the CI in regularised model seem to provide nominal coverage. The trick would be in determining when that happens? Would that be possible based on the unpenalized effect size and magnitude of the tuning parameter?

bbolker commented 2 years ago

Continuing in the Bayes/freq crossover tradition: we know that Bayesian models are well-calibrated, by definition; that is, if we draw parameters from the prior (and the sampling/estimation method isn't broken), then we will get exact coverage. It's a little slippery to define this properly in frequentist-world, but my take has always been that if the form of the penalization implicitly matches the effect size spectrum then things will work well.

Talts, Sean, Michael Betancourt, Daniel Simpson, Aki Vehtari, and Andrew Gelman. “Validating Bayesian Inference Algorithms with Simulation-Based Calibration.” ArXiv:1804.06788 [Stat], October 21, 2020. http://arxiv.org/abs/1804.06788.

BertvanderVeen commented 2 years ago

It might be worth having a look at Wood (2006) on this: https://onlinelibrary.wiley.com/doi/full/10.1111/j.1467-842X.2006.00450.x. Mgcv by default makes use of this 'Bayesian estimated covariance matrix of the parameter estimators'. See also https://onlinelibrary.wiley.com/doi/full/10.1111/j.1467-9469.2011.00760.x.