Closed turion closed 1 year ago
The test code in question is:
describe "Integrator Variance" do
prop "variance numerically" $
\mean var ->
var > 0 ==> property $ TestIntegrator.normalVariance mean (sqrt var) ~== var
I'd suspect floating point rounding issues.
@reubenharry any ideas?
If there were no changes, I'm slightly confused, because I believe the integrator is deterministic. (Oh but I guess quickcheck isn't). Does the case it identified consistently fail? The negative variance is...odd.
It's not a huge worry, because the integrator is mainly just a toy implementation, and isn't used in any real inference algorithm, but maybe worth double checking the failure case.
This might even lead to monad-bayes
getting marked broken in nixpkgs
, see https://github.com/NixOS/nixpkgs/pull/236431
Ah, this might be due to a better QuickCheck generator that revealed test failures that were false negatives:
Or similar to https://github.com/haskell/vector/issues/460 it might be that the new generator produces more values for which we experience overflows.
I can't reproduce it with QuickCheck 2.14.3 yet.
Worth noting that the Integrator is really just a toy implementation, which is never used in any real inference algorithm in Monad-Bayes. Quickest thing would be just to remove these tests. Or even just remove the Integrator altogether (although it's quite nice for graphing distributions).
Still it would be great to understand what's going wrong. If we can't solve a test problem with a toy implementation, the real inference algorithms will be even harder to debug.
Maybe var > 0
is the issue, and we should have something like "var
is significantly above 0". For example, var > 0 && not (var ~== 0)
.
See https://github.com/tweag/monad-bayes/actions/runs/5001514340/jobs/8960326116#step:6:34973
This test usually passes, but it failed here without any code changes.