Sbc_rank in "A Principled Bayesian Workflow"

[Paul-Yuchao-Dong]

Thank you @betanalpha for this informative and hands on tutorial on the workflow!

I am confused and had been thinking for days on the sbc_rank of prior draw with respect to thinned posterior draws. the distribution of these rankings are supposed to be uniform, and for good models the result seems to be uniform.

My question, though, is why isn't it more concentrated? the code seems to calculate the rank of the true parameter that were used to generate the date against the fitted draws from posterior. If the posterior is well-fitted, wouldn't we expect to recover this parameter?

of course the choice of prior matters, in this case it means the posterior would have a tendency to go for smaller values.

then based on the code, I would expect the mode of the rank to be smaller than 250 (4000/8/2), yet the result was uniform.

Is there any resources that I can read to clear my head?

thanks

[betanalpha]

If the posterior is well-fitted, wouldn't we expect to recover this parameter?

No! There is no guarantee that any given posterior will “cover” the true parameters. The only guarantee that we have is that the “average” posterior equals the prior in this simulation contexts. For a much more thorough discussion I recommend reading through the SBC paper, https://arxiv.org/abs/1804.06788 https://arxiv.org/abs/1804.06788.

[Paul-Yuchao-Dong]

Thank you for making these research public! Your SBC paper is impressive.

Due to my limited understanding of mathematical notation, I cannot breakthrough your rigorous proof in Appendix B. (Like how is $ P(\tilde{f}) $ defined and why did binomial probability mass function emerge in the first step? Also it seems the font of P changed in the second step, that seems to have simplified things a lot, but is it because of a type change?)

However, your words in section 4.1 make intuitive sense to me. if the fitted model were good, then it would have drawn samples from the posterior distribution of the parameter. In the prior checking step, the data generating process is known, so the parameter used to generate the fake (simulated) data can be seen as drawn from the same distribution.

Then if we assume the independent and identical draws, the rank statistic would be close to uniform by symmetry.

I know my understanding is not enough for me to understand the binomial error that is allowable in the distribution of rank statistics, but I would read on and find out more. Thanks!