The following has been a thorn in my side in another project and here is a minimal failing example:
prop_discardCoverage :: Property
prop_discardCoverage = forAll (sized $ \ n -> pure n) $ \ x -> cover 10 True "label" $ x /= 99 ==> True
What happens is that we get a failure like:
$> quickCheck $ checkCoverage prop_discardCoverage
*** Gave up! Passed only 99 tests; 1 discarded tests (100% label).
when we expect the property to succeed. The reason for this is that on the 99th test (with size 98) we run addCoverageCheck which adds a once around the property. In the next test we are "unlucky" (in the real world this test is also flaky!!) and hit the discarded test, causing the whole test to be discarded because we discarded a once.
The following has been a thorn in my side in another project and here is a minimal failing example:
What happens is that we get a failure like:
when we expect the property to succeed. The reason for this is that on the 99th test (with size 98) we run
addCoverageCheckwhich adds aoncearound the property. In the next test we are "unlucky" (in the real world this test is also flaky!!) and hit the discarded test, causing the whole test to be discarded because we discarded aonce.I have a fix for this in the pipeline.