stan-dev / posteriordb

Database with posteriors of interest for Bayesian inference
180 stars 36 forks source link

Add SBC structure to posteriordb #111

Open MansMeg opened 4 years ago

MansMeg commented 4 years ago

Include SBC in the psoteriordb. How should that be included for a model to be . tested using SBC. What additional stuff would be needed.

See SBC by Bob Carpenter and Ben Goodridge

eerolinna commented 4 years ago

Do you have any ideas how this should be done?

MansMeg commented 4 years ago

I need to think how to do this in the best way. Either adding simulation from the prior or a separate Stan code. I dont know SBC well enough yet.

eerolinna commented 4 years ago

To run SBC we essentially need a way to obtain prior and prior predictive samples. I suggest we use the model implementations approach that we talked about in #112

For Stan it could mean that the model implementation file would be

{
  "code": "models/stan/blr.stan",
  "likelihood": "models/stan/blr_likelihood.stan",
  "prior_and_predictive": "models/stan/blr_prior.stan"
}

where the prior_and_predictive would be a stan program that draws prior and prior predictive samples.

For pymc I think we don't need anything extra so the model implementation file could be

{
  "code": "models/pymc/blr.py"
}

This is because pymc has API to draw prior predictive samples for any model

prior_predictive_draws = pymc.sample_prior_predictive(n_samples, model=model)
MansMeg commented 4 years ago

That looks good!