cuplv / cuanto

Apache License 2.0
1 stars 1 forks source link

FixFunSpec seems to fail on some rare ScalaCheck cases #71

Open octalsrc opened 6 years ago

octalsrc commented 6 years ago

When running the "Push" Travis tests for japron-fresh, we got a failure in the FixFunSpec. The failure did not appear during the second "PR" test run, which makes me think that this failure is non-deterministic (meaning it might have been hanging around a while!).

The particular test uses ScalaCheck to generate random test cases, so there must be some form of expression that ScalaCheck produces very rarely which sends the eval function into infinite recursion.

[info] FixFunSpec:
[info] plusexpr.pretty
[info] - should pretty-print N(1) to 1
[info] - should pretty-print Neg(N(2)) to (-2)
[info] - should pretty-print Plus(N(1),N(2)) to (1 + 2)
[info] - should pretty-print Neg(Plus(N(2),N(3))) to (-(2 + 3))
[info] plusexpr.eval
[info] - should evaluate N(1) to 1
[info] - should evaluate Neg(N(2)) to -2
[info] - should evaluate Plus(N(1),N(2)) to 3
[info] - should evaluate Neg(Plus(N(2),N(3))) to -5
[info] - should not crash on arbitrary expressions *** FAILED ***
[info]   StackOverflowError was thrown during property evaluation.
[info]     Message: "None"
[info]     Occurred when passed generated values (
[info]   
[info]     )
octalsrc commented 6 years ago

We need to edit the sometimes-failing FixFunSpec test to log each random case that it runs on, and then run it lots of times until we see the failure again (and now know what the failing example is).