pholser / junit-quickcheck

Property-based testing, JUnit-style
MIT License
955 stars 121 forks source link

Function generators always generate the same function #474

Closed non closed 9 months ago

non commented 1 year ago

Here's an example test that demonstrates the problem:

package demo;

import com.pholser.junit.quickcheck.Property;
import com.pholser.junit.quickcheck.runner.JUnitQuickcheck;
import org.junit.runner.RunWith;
import java.util.function.Function;

import static org.junit.Assert.assertTrue;

@RunWith(JUnitQuickcheck.class)
public class DemoProperties {
    @Property public void differentFns(Function<Integer, Integer> f) {
        System.out.println("f(0) = %d".formatted(f.apply(0)));
        assertTrue(f.apply(0) % 2 == 0);
    }
}

On my machine this erroneous test always passes (the value generated for f(0) is always 58927336). I would expect half of the generated functions to produce an odd output for f(0).

The problem is that when generating a function, the seed is ignored. The lambda generator hashes its inputs to produce outputs, which means all generated functions work the same way. The easiest fix would be to combine the original generator seed (currently unused) with the hashed arguments to produce an output. This would ensure that two generated function would likely behave differently for the same input.

pholser commented 1 year ago

Thanks for this! Good catch

pholser commented 1 year ago

@non Have a look at the above PR -- let me know how this works for you. Thanks again!

pholser commented 12 months ago

@non If you have a moment, check out the PR mentioned above, and close the issue if it's resolved for you. Thanks!

pholser commented 9 months ago

Closing as fix should be on HEAD. Re-open if it doesn't work for you.