betanalpha / knitr_case_studies

Inference case studies in knitr
163 stars 94 forks source link

Inversed definition of biased SBC histograms? #37

Closed fweber144 closed 3 years ago

fweber144 commented 3 years ago

The case study "Towards A Principled Bayesian Workflow" presents two possible definitions of SBC ranks:

$$ \rho = \sharp \left{ \tilde{\theta} < \tilde{\theta}'_{r} \right}. $$

and

$$ \rho = \sharp \left{ \tilde{\theta} > \tilde{\theta}'_{r} \right} $$

with $\tilde{\theta}$ being the true parameter value and $\tilde{\theta}'_{r}$ being a posterior draw (I assume). For the first definition, you say

Similarly if the estimated posterior is biased low then the prior ranks will be biased high and the SBC histogram will concentrate at the right boundary.

For the second definition, you say

then a low bias in the posterior fit will manifest in a SBC histogram that concentrates at the left boundary.

My question is if these two interpretations shouldn't be inversed.

betanalpha commented 3 years ago

Yes, they should, thanks! In particular I'm using the opposite convention from the SBC paper, hence the flipped interpretation.

Will add to the to-fix list.

betanalpha commented 3 years ago

Addressed in aeab31509b8e37ff05b0828f87a3018b1799b401.