Allow to Skip Current Generation Attempt in Provider Methods

[jlink]

Testing Problem

Motivated by example in https://github.com/jlink/jqwik/issues/405#issuecomment-1310041452.

Sometimes you want to skip the current generation attempt because previously generated values that are flat-mapped don't allow generation. Example:

@Provide
Arbitrary<MyObject> myObjects(@ForAll int size, @ForAll String code) {
    if (size == 42) {
        // skip generation here because size 42 does never work
    }
    try {
        return just(new MyObject(size, code));
    } catch (IllegalArgumentException e) {
        // skip generation because some combination of size and code is not allowed
    }
}

Suggested Solution

Introduce something like Arbitraries.skip() or Arbitraries.fail() which will proceed to next generation attempt. This only makes sense (I assume) if there's some flat-mapping going on.

Discussion

• Many of those cases can (better) be solved by filtering out certain parameter combinations.
• Could be related to https://github.com/jlink/jqwik/issues/335
• Would Assume.that(..) be a viable option instead of Arbitraries.skip()?

[adam-waldenberg]

In some cases (like in our case), the exception is coming from a dependency. So we really don't know all the corner cases that trigger it. So manual filtering and/or Assume doesn't really work.

[jlink]

So manual filtering and/or Assume doesn't really work.

try {
   ...
} catch (Throwable t) {
   Assume.that(false);
}

not nice, but what in this world is...

[adam-waldenberg]

Just realized something.... Maybe some kind of mechanism to rerun the provider on an exception would be cleaner? Something like this.

@Provide(repeatOnException = { /* List of exceptions */ })
Arbitrary<MyObject> myObjects(@ForAll int size, @ForAll String code) {
    if (size == 42) {
        // skip generation here because size 42 does never work
    }

    /* Instead of catching it we just let it fall through */
    return new MyObject(size, code);
}

That makes the code even cleaner and also makes it quite clear what is happening. No need to return any empty values or crap like that.

[jlink]

Maybe some kind of mechanism to rerun the provider on an exception would be cleaner?

Worth pondering.

[jlink]

Turns out that @Provide(ignoreException = { /* List of exceptions */ }) just for methods annotated with @Provide would be much simpler to implement than a generic Arbitraries.skip() feature.

@adam-waldenberg I assume it would serve your needs since you brought that option up?

[adam-waldenberg]

Yes. Would that end up with the provider executing again, or would it still return a null value back to the test ? That is, will I still need my @NotNull filter?

[jlink]

Would just skip generation attempts with specified exception(s). There’s already ˋArbitrary.ignoreException(..)ˋ which does what we want. It just has to be applied on the arbitrary resulting from the provider method.

[adam-waldenberg]

Sounds like a plan then.

[jlink]

@Provide(ignoreExceptions = {..}) is now available in 1.7.2-SNAPSHOT.

Documentation for this is still missing, though.

[jlink]

Added a paragraph about the new feature: https://jqwik.net/docs/snapshot/user-guide.html#ignoring-exceptions-in-provider-methods

[jlink]

1.7.2 has been released

[adam-waldenberg]

:partying_face: