Running Bayesian sensitivity analysis

[jeromyanglim]

• What is the general strategy for running a sensitivity analysis of Bayesian estimation?

[jeromyanglim]

• features to explore
  • difference between posterior mean and true parameter value
  • does discrepancy between posterior mean and true parameter value decrease as sample size increases
• factors
  • true parameters
  • true model generating sample
  • sample size
  • bayesian model (likelihood and prior)
  • model fitting options (number of chains, adaptation iterations, burn-in iterations, monitor iterations)
• Broad questions
  • Does the model fitting procedure recover true values:
  • Reasonable standard error: Are deviations between posterior and true reasonable given sampling error?
  • Absence of bias: Does the mean of estimates of the poterior mean approach the true value?
  • Under what conditions does model fitting break down?

Chapter 8 of Gelman and Hill (2007)

This chapter is titled "simulation for checking statistical procedures and model fits". It covers:

• fake data simulation: simulating data and seeing whether model produces appropriate estimates given true values
• predictive simulation: simulate data from model and see whether simulated data adequately matches actual data