Computing diagnostics during inference would allow users to monitor how well their model is performing without having to wait until thousands of samples were generated. In particular, it would be interesting to be able to monitor divergences and Rhat in real time.
Divergences are detected during the inference and is straightforward to report. (non-split) Rhat and ESS should be computable as the chains are being computed. Both can rely on the Welford algorithm already used to adapt the mass matrix.
A list of references that I will keep adding here:
Computing diagnostics during inference would allow users to monitor how well their model is performing without having to wait until thousands of samples were generated. In particular, it would be interesting to be able to monitor divergences and Rhat in real time.
Divergences are detected during the inference and is straightforward to report. (non-split) Rhat and ESS should be computable as the chains are being computed. Both can rely on the Welford algorithm already used to adapt the mass matrix.
A list of references that I will keep adding here: