QuickCheck: allow to print/specify seed for a failure without having to replay through previous successes

[amesgen]

If we find a property test failure after a lot of iterations, reproducing it is annoying as we have to replay all successful iterations to get to the failing test case. This is a bit annoying:

• major: reproducing the failure can take a long time when testing with lots of iterations and/or long test durations per test (especially true for long-running nightly tests)
• minor: ad-hoc debugging statements via eg Debug.Trace are also printed for all previous successful cases

Also see https://github.com/hedgehogqa/haskell-hedgehog/pull/454 for a recent Hedgehog PR implementing this (and more).

I think this should be simple to implement in tasty-quickcheck as QuickCheck exposes the necessary seed in its Result:

data Result
  = ...
  | Failure
    { ...
    , usedSeed        :: QCGen
      -- ^ What seed was used
    , usedSize        :: Int
      -- ^ What was the test size
    ...
    }
...

data Args
  = Args
  { replay          :: Maybe (QCGen,Int)
    -- ^ Should we replay a previous test?
  , ...
  }

On failure, (usedSeed, usedSize) would be printed, and it could be specified via a new flag --quickcheck-exact-replay (name very tentative).

Happy to open a PR if this sounds like a useful addition.

[Bodigrim]

Overall makes sense to me, but hard to judge further without looking at a PR (ideally with tests).

[facundominguez]

I contributed a PR with a test.