nick8325 / quickcheck

Automatic testing of Haskell programs.
Other
706 stars 119 forks source link

Witnesses of labelledExamples? #395

Open stevana opened 3 months ago

stevana commented 3 months ago

With #376 closed we can get access to counterexamples:

>>> [Wit x] <- fmap witnesses . quickCheckResult $ \ x -> witness x $ x == (0 :: Int)
*** Failed! Falsified (after 2 tests):
1
>>> x
1
>>> :t x
x :: Int

I was wondering if we can extend this to labelledExamplesResult to get access to the examples?

I'd like to be able to do something like:

>>> [("five", Wit i), ("ten", Wit j)] <- fmap examples . 
  labelledExamplesResult $ \x -> 
    classify (x == 5) "five" $ 
    classify (x == 10) "ten" True
>>> i
5
>>> j
10
MaximilianAlgehed commented 2 months ago

I suppose the type you'd want for examples is something like [(String, [Witness])] to deal with the situation where you have multiple arguments to you property? (As per the example in Test.QuickCheck.Features).

To implement this you'd need to extend the State type to collect the examples, and you'd need to poke around in the Result type. Completely doable.

The thing that's missing from your example, by the way, are the calls to witness as there are not put in automatically by QuickCheck (as doing so would require having Typeable everywhere and that's just horrible!).

>>> [("five", [Wit i]), ("ten", [Wit j])] <- fmap examples . 
  labelledExamplesResult $ \x -> 
    witness x $
    classify (x == 5) "five" $ 
    classify (x == 10) "ten" True
>>> i
5
>>> j
10
MaximilianAlgehed commented 2 months ago

If you're interested in diving into the source code of QuickCheck and doing this yourself I'd be very happy to review the PR!