nick8325 / quickcheck

Automatic testing of Haskell programs.
Other
706 stars 119 forks source link

`DISCARD` exception shows up user-side #392

Closed MaximilianAlgehed closed 3 months ago

MaximilianAlgehed commented 3 months ago

We observe the following in ghci:

$> generate (discard :: Gen Int)
$> a <- generate (discard :: Gen Int)
$> a
*** Exception: DISCARD. You should not see this exception, it is internal to QuickCheck.
$> sample (discard :: Gen Int)
*** Exception: DISCARD. You should not see this exception, it is internal to QuickCheck.

Note also the different strictness behaviours of generate and sample (becuase sample prints the value). The problem here is not the failure per se, but the fact that it gladly fails with "You should not see this exception". "Fixing" the strictness of generate I can imagine being a bit controversial, but we could at least catch the exception and print something more reasonable than "You should not see this exception" and in sample it should be straight-forward.

MaximilianAlgehed commented 3 months ago

Furthermore, the really problematic part of this is the behaviour of sample:

$> sample (oneof [discard, pure 1] :: Gen Int)
*** Exception: DISCARD. You should not see this exception, it is internal to QuickCheck.
$> sample (oneof [discard, pure 1] :: Gen Int)
1
*** Exception: DISCARD. You should not see this exception, it is internal to QuickCheck.

It should be giving me plenty of examples but instead it fails with an exception. I would have expected to see something like:


1
1
1
1
<Some message telling me it was discarded>
<Some message telling me it was discarded>
1
1
<Some message telling me it was discarded>
MaximilianAlgehed commented 3 months ago

I'm working on a fix for this on a branch where I'm fixing it for sample. Fixing it for generate is tricker on account of the fact that we probably don't want to make generate strict. What we can do is hide the internal discard error at least.