jqwik-team / jqwik

Property-Based Testing on the JUnit Platform
http://jqwik.net
Eclipse Public License 2.0
578 stars 64 forks source link

Arbitrary proiders registered under a Domain don't seem to support @ForAll properly #407

Closed adam-waldenberg closed 2 years ago

adam-waldenberg commented 2 years ago

Testing Problem

The following code:

public class CustomDomain extends DomainContextBase {
    @Provide
    public Arbitrary<MyType> provideMyType(@ForAll int number) {
        return Arbitraries.of(new MyType());
    }
}

public class MyTest {
    @Property
    @Domain(CustomDomain.class)
    public void shouldMatch(@ForAll MyType myType) {
        /* Some testring logic */
    }
}

This throws an exception complaining that an arbitrary for parameter type int cannot be found - something I am assuming should work. At least that is what I would expect.

Current Workaround

While not ideal - current workaround is to use inheritance (rather than a Domain annotation) and share the provider that way.

jlink commented 2 years ago

The feature works almost as you expect. Let me explain: Domains are conceived to give you perfect control about how objects of a certain domain (think accounting, sales, ...) are being created. That means, that by default they do not inherit the global context, ie. all the default and globally registered arbitrary providers.

To make your example work, you'll have to tell jqwik to use the global context as well. You can do that in one of two ways. Either declare it at the specific domain:

@Domain(DomainContext.Global.class)
public class CustomDomain extends DomainContextBase {
    @Provide
    public Arbitrary<MyType> provideMyType(@ForAll int number) {
        return Arbitraries.of(new MyType());
    }
}

Or at the place where you use the domain:

@Domain(DomainContext.Global.class) // alternatively on the test container class
public void shouldMatch(@ForAll MyType myType) {
    /* Some testing logic */
}

That said, domains are not considered to be the best way for reusing provider methods. Consider arbitrary suppliers instead.

And one more thing on using @ForAll parameter in provider methods. Under the hood, they will always fall back to flat mapping over the arbitrary generating the injected parameter. You pay for flat-mapping with rather non-optimal shrinking. If you really need it depends on how you use the parameter. In your example, flat-mapping may be warranted, if the injected number is being used for configuring the resulting arbitrary, e.g.

@Provide
public Arbitrary<List<MyType>> provideMyTypes(@ForAll int number) {
    return Arbitraries.create(() -> new MyType()).list().ofSize(number);
}

If you just want to use number in a composed value, I'd recommend plain mapping, e.g.:

@Provide
public Arbitrary<MyType> provideMyType() {
    return Arbitraries.integers().map (number -> new MyType(number));
}
adam-waldenberg commented 2 years ago

Got it! Sneaky. I see it's in one of the examples on the User guide - but it probably needs to be documented a little better.

jlink commented 2 years ago

but it probably needs to be documented a little better.

I agree. Will put it on my todo list for the next release.