search

nick8325 / quickcheck

Automatic testing of Haskell programs.
Other
713 stars 119 forks source link

Pretty-show-ing the counterexample. #328

Open NorfairKing opened 3 years ago

NorfairKing commented 3 years ago

As part of my work on https://github.com/NorfairKing/sydtest, I'd like to ppShow the generated counterexample. The failingTestCase seems to have hard-coded show. Is there any way around this? Is there any chance you might expose the failing test case as a Show box instead of a string?

MaximilianAlgehed commented 5 months ago

The tricky thing here is that if you just expose a Box :: forall a. Show a => a -> Box in the counterexample there is nothing you can do with that other than show it (thanks parametricity). So, to get around that you would need to have a Box :: forall a. c a => a -> Box c as the thing you expose instead. However, that becomes tricky to deal with for other reasons - now all of a sudden we have this c parameter floating around that needs to be pinned down in e.g. forAll.

Over in #376 we have discussed briefly putting Typeable on forAll in order to automatically expose a Counterexample :: forall a. Typeable a => a -> Counterexample box. However, this doesn't immediately help with hooking in an arbitrary prettyprinter because you'd still need to derive MyPretty a from Typeable a - which requires having a run-time table of dictionaries (basically lookupDict :: forall a. Typeable a => Maybe (Dict MyPretty a) for Dict c a :: c a => Dict c a.

A workaround for this whole mess is defining a WithPretty :: MyPretty a => a -> WithPretty a modifier with all the relevant Arbitrary instances and rewriting your property using WithPretty in a similar manner to how you would do with Positive or Small. With #376 in place in one format or another that would then give you the ability to work with that counterexample and display it in some reasonable way in your testing framework.

In short, there is no obvious design that makes it easy to hook up arbitrary prettyprinters without changing the interface of QuickCheck quite a lot. But there are user side solutions today and more advanced solutions once #376 has been implemented.