On a failure, it would be nice if proptest printed a string that could be used to more immediately replay exactly the failing test (i.e. on the very first try).
For example, the JS fast-check library will print something like:
You can then either paste that into the code or set an environment variable so that the very next run of the test is the same as the one that caused the problem.
I know the note about serializing vs picking the seed, but I feel like fast-check picks a middle where it saves the set of operations to perform against the seed to reach the relevant test (i.e. you still evaluate all the strategies, but you skip invoking the test case until you reach the appropriate nth invocation). Additionally, if your RNG supports it (e.g. PCG) & you know that you're going to discard the value anyway, you could skip generating any actual data & just move the RNG state forward "as-if" you had actually generated the relevant value for the strategy.
On a failure, it would be nice if proptest printed a string that could be used to more immediately replay exactly the failing test (i.e. on the very first try).
For example, the JS fast-check library will print something like:
You can then either paste that into the code or set an environment variable so that the very next run of the test is the same as the one that caused the problem.
I know the note about serializing vs picking the seed, but I feel like fast-check picks a middle where it saves the set of operations to perform against the seed to reach the relevant test (i.e. you still evaluate all the strategies, but you skip invoking the test case until you reach the appropriate nth invocation). Additionally, if your RNG supports it (e.g. PCG) & you know that you're going to discard the value anyway, you could skip generating any actual data & just move the RNG state forward "as-if" you had actually generated the relevant value for the strategy.