search

nick8325 / quickcheck

Automatic testing of Haskell programs.
Other
713 stars 119 forks source link

within not triggering failure #276

Closed BebeSparkelSparkel closed 4 years ago

BebeSparkelSparkel commented 4 years ago

within does not cause the test to fail when applied before quickCheckWith but works when applied after.

fit "within" $
    quickCheckWith stdArgs $
    \(s :: String) -> within 1 $ s `shouldBe` s
*** Failed! Timeout (after 22 tests):
    within

Finished in 0.0051 seconds
1 example, 0 failures

Test suite passed

This may be an hspec issue but it seems to happen from QuickCheck function ordering.

BebeSparkelSparkel commented 4 years ago

Without quickCheckWith modifying the test, within fails properly

fit "within" $
    property $ \(s :: String) -> within 1 $ s `shouldBe` s
phadej commented 4 years ago

Can you provide a reproduce case without depending on hspec, i.e. using only QuickCheck?

sol commented 4 years ago

@BebeSparkelSparkel quickCheckWith is not what you want to use here. Look at https://hackage.haskell.org/package/hspec-2.7.1/docs/Test-Hspec-QuickCheck.html if you want to modify QuickCheck args.

I don't plan to elaborate on this, but in short, quickCheckWith is of type IO () and does not throw and exception on failure. There is no sensible way for Hspec to discover the test outcome if you use quickCheckWith.

BebeSparkelSparkel commented 4 years ago

@sol I understand now. Thanks