jqwik-team / jqwik

Property-Based Testing on the JUnit Platform
http://jqwik.net
Eclipse Public License 2.0
551 stars 63 forks source link

Allowing Sampling of Arbitraries with Injected Seed #581

Open gypsydave5 opened 2 days ago

gypsydave5 commented 2 days ago

Testing Problem

I like using random data in my tests as often as possible - so if a test fixture doesn't need to be a particular value for the test, I set it to some random instance.

jqwik Arbitraries are great for this! Just call Arbitrary.sample() and away we go, they compose beautifully. Really enjoy it.

However - because I can't control the randomness that sample is using, it means that it's harder to make the tests repeatable (I'd have to output some representation of the sampled object).

Suggested Solution

I was originally thinking that I'd want to inject the seed directly to sample, but taking a look at the JqwikSession api, and the two methods in the TODO file setRandomSessionSeed(), getRandomSessionSeed(), can I infer that this is the intended direction of travel - so set up my session, use it in the test(s), output the seed on failure, and inject it when I want to repeat?

Discussion

I'm guessing the solution is for me to shut up and wait / offer to help out? 😄

jlink commented 2 days ago

You can do it with a bit more verboseness:

Arbitrary<String> strings = Arbitraries.strings().withCharRange('a', 'z').ofMinLength(1);
Random random = new Random(42);
JqwikSession.run(() -> {
    strings.generator(1000)
           .stream(random).limit(10)
           .map(Shrinkable::value)
           .forEach(System.out::println);
});

That said, I'll set the session methods for JqwikSession on the TODO list for 1.9.1 since there's nothing urgent there and all big features will have to wait for Jqwik 2.