The current mcse_sd assumes normality. More accurate mcse_sd for non-normally distributed variables can be obtained using moments of moments and first order Taylor series approximation as described by Kenney and Keeping (1951, p. 141) (Mathematics of Statistics, Part two, 2nd edition). I also made simulations to find out better ESS choice. The previous ess_sd was also ad-hoc and was a conservative choice that that underestimated ESS. New ess_mean(abs(sims-mean(sims))) is less ad-hoc, as it is more related to the actual computation for sd, and empirically it provides a good ESS with increasing S.
The new computation has marginally very low bias, and the remaining excess conditional error is unavoidable due to the structure of the estimation problem: a) with small S, observed sd has high variation, b) when observed sd is smaller than asymptotic sd, MCSE which is based on sd is also smaller, and then standardized errors for small sd are big, and these dominate the error plots.
The following plots show the root mean square of the conditional standardized errors for marginally normal(0,1) and marginally Chi(1) distributed variables with underlying correlation from AR(1, phi) process. The same simulation were used in Rhat online postscript appendix https://avehtari.github.io/rhat_ess/ess_comparison.html
avehtari
commented
2 years ago
The current mcse_sd assumes normality. More accurate mcse_sd for non-normally distributed variables can be obtained using moments of moments and first order Taylor series approximation as described by Kenney and Keeping (1951, p. 141) (Mathematics of Statistics, Part two, 2nd edition). I also made simulations to find out better ESS choice. The previous ess_sd was also ad-hoc and was a conservative choice that that underestimated ESS. New ess_mean(abs(sims-mean(sims))) is less ad-hoc, as it is more related to the actual computation for sd, and empirically it provides a good ESS with increasing S.
The new computation has marginally very low bias, and the remaining excess conditional error is unavoidable due to the structure of the estimation problem: a) with small S, observed sd has high variation, b) when observed sd is smaller than asymptotic sd, MCSE which is based on sd is also smaller, and then standardized errors for small sd are big, and these dominate the error plots.
The following plots show the root mean square of the conditional standardized errors for marginally normal(0,1) and marginally Chi(1) distributed variables with underlying correlation from AR(1, phi) process. The same simulation were used in Rhat online postscript appendix https://avehtari.github.io/rhat_ess/ess_comparison.html