I got irritated by a stochastic CI failure on how different the sample variance is from sampling from `SafeNegativeBinomial` vs `NegativeBinomial` so I made it more principled test based on the std of the variance estimator.
The idea here is that we should relate a "bad" failure that indicates a high probability of a miscoded sampler to the properties of the estimator being tested. We've had intermittent failure on doing this for NegativeBinomial sampling (iirc) but thats to be expected because the variance of each draw is fairly high and therefore the std of the population variance estimator is fairly high and is competitive against the $O(1/\sqrt{n})$ factor.
seabbs
commented
1 month ago
Originally posted by @SamuelBrand1 in https://github.com/CDCgov/Rt-without-renewal/issues/418#issuecomment-2288686092
The idea here is that we should relate a "bad" failure that indicates a high probability of a miscoded sampler to the properties of the estimator being tested. We've had intermittent failure on doing this for
NegativeBinomialsampling (iirc) but thats to be expected because the variance of each draw is fairly high and therefore the std of the population variance estimator is fairly high and is competitive against the $O(1/\sqrt{n})$ factor.