haskell / statistics

A fast, high quality library for computing with statistics in Haskell.
http://hackage.haskell.org/package/statistics
BSD 2-Clause "Simplified" License
300 stars 68 forks source link

Exact tests for some distributions #199

Closed lorinder closed 1 year ago

lorinder commented 1 year ago

This implements a few tests of some distributions against exact, rational implementations. I used these to find the accuracy bugs for which I recently reported fixes.

The obvious advantage of having such tests is that they compare against a benchmark distribution, so if integrated, it could guard against future regressions. It also makes it possible to get good information accuracy of the distributions. One possible drawback is that some of the tests are relatively slow; in particular the one for the binomial distribution. Personally, I think the few seconds there are bearable and worth the tradeoff. It would be possible, however, to speed things up by choosing smaller instances for that test. (The tradeoff being here, that the test would no longer be as sharp.)

Shimuuar commented 1 year ago

Yes that's great!

P.S. It's possible to use type application together with ScopedTypeVariables in order to write pmfMatch @a tol instead of (pmfMatch tol) :: TestCase a -> Bool. With AllowAmbiguousTypes it possible to dispence with Tag as well: distTests :: forall a. (...) => String -> Double -> TestTree. But that leads to worse error messages.