1.8.0 contracts regression

[SimY4]

Testing Problem

interface EqualityContract<T> {
  @Property
  default boolean reflexivityEq(@ForAll("anyT") T value) {
    return Objects.equals(value, value);
  }
  Arbitrary<T> anyT();
}

class MyTypeTest implements EqualityContract<MyType> {
  @Provide("anyT")
  Arbitrary<MyType> anyT() { ... }
}

fails with:

net.jqwik.api.CannotFindArbitraryException: Cannot find an Arbitrary [anyT] for Parameter of type [T]
    at net.jqwik.engine.properties.RandomizedShrinkablesGenerator.resolveArbitraries(RandomizedShrinkablesGenerator.java:152)
    at net.jqwik.engine.properties.RandomizedShrinkablesGenerator.resolveEdgeCases(RandomizedShrinkablesGenerator.java:124)
    at net.jqwik.engine.properties.RandomizedShrinkablesGenerator.listOfEdgeCases(RandomizedShrinkablesGenerator.java:93)
    at net.jqwik.engine.properties.RandomizedShrinkablesGenerator.forParameters(RandomizedShrinkablesGenerator.java:28)
    at net.jqwik.engine.execution.CheckedProperty.createRandomizedShrinkablesGenerator(CheckedProperty.java:236)
    at net.jqwik.engine.execution.CheckedProperty.createForAllParametersGenerator(CheckedProperty.java:176)
    at net.jqwik.engine.execution.CheckedProperty.createGenericProperty(CheckedProperty.java:144)
    at net.jqwik.engine.execution.CheckedProperty.check(CheckedProperty.java:67)
    at net.jqwik.engine.execution.PropertyMethodExecutor.executeProperty(PropertyMethodExecutor.java:90)
    at net.jqwik.engine.execution.PropertyMethodExecutor.executeMethod(PropertyMethodExecutor.java:69)
    at net.jqwik.engine.execution.PropertyMethodExecutor.lambda$execute$0(PropertyMethodExecutor.java:49)
    at net.jqwik.api.lifecycle.AroundPropertyHook.lambda$static$0(AroundPropertyHook.java:46)
    at net.jqwik.engine.execution.lifecycle.HookSupport.lambda$wrap$0(HookSupport.java:26)
    at net.jqwik.api.lifecycle.PropertyExecutor.executeAndFinally(PropertyExecutor.java:39)
    at net.jqwik.engine.hooks.lifecycle.PropertyLifecycleMethodsHook.aroundProperty(PropertyLifecycleMethodsHook.java:56)
    at net.jqwik.engine.execution.lifecycle.HookSupport.lambda$wrap$1(HookSupport.java:31)
    at net.jqwik.engine.execution.lifecycle.HookSupport.lambda$wrap$0(HookSupport.java:26)
    at net.jqwik.engine.hooks.statistics.StatisticsHook.aroundProperty(StatisticsHook.java:37)
    at net.jqwik.engine.execution.lifecycle.HookSupport.lambda$wrap$1(HookSupport.java:31)
    at net.jqwik.engine.execution.lifecycle.HookSupport.lambda$wrap$0(HookSupport.java:26)
    at net.jqwik.engine.hooks.lifecycle.AutoCloseableHook.aroundProperty(AutoCloseableHook.java:13)
    at net.jqwik.engine.execution.lifecycle.HookSupport.lambda$wrap$1(HookSupport.java:31)
    at net.jqwik.engine.execution.PropertyMethodExecutor.execute(PropertyMethodExecutor.java:47)
    at net.jqwik.engine.execution.PropertyTaskCreator.executeTestMethod(PropertyTaskCreator.java:166)
    at net.jqwik.engine.execution.PropertyTaskCreator.lambda$createTask$1(PropertyTaskCreator.java:51)
    at net.jqwik.engine.execution.lifecycle.CurrentDomainContext.runWithContext(CurrentDomainContext.java:28)
    at net.jqwik.engine.execution.PropertyTaskCreator.lambda$createTask$2(PropertyTaskCreator.java:50)
    at net.jqwik.engine.execution.pipeline.ExecutionTask$1.lambda$execute$0(ExecutionTask.java:31)
    at net.jqwik.engine.execution.lifecycle.CurrentTestDescriptor.runWithDescriptor(CurrentTestDescriptor.java:17)
    at net.jqwik.engine.execution.pipeline.ExecutionTask$1.execute(ExecutionTask.java:31)
    at net.jqwik.engine.execution.pipeline.ExecutionPipeline.runToTermination(ExecutionPipeline.java:82)
    at net.jqwik.engine.execution.JqwikExecutor.execute(JqwikExecutor.java:46)
    at net.jqwik.engine.JqwikTestEngine.executeTests(JqwikTestEngine.java:70)
    at net.jqwik.engine.JqwikTestEngine.execute(JqwikTestEngine.java:53)
    at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:107)
    at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:88)
    at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:54)
    at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:67)
    at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:52)
    at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:114)
    at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:86)
    at org.junit.platform.launcher.core.DefaultLauncherSession$DelegatingLauncher.execute(DefaultLauncherSession.java:86)
    at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.processAllTestClasses(JUnitPlatformTestClassProcessor.java:118)
    at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.access$000(JUnitPlatformTestClassProcessor.java:93)
    at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor.stop(JUnitPlatformTestClassProcessor.java:88)
    at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.stop(SuiteTestClassProcessor.java:62)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:568)
    at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
    at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
    at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
    at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
    at jdk.proxy2/jdk.proxy2.$Proxy5.stop(Unknown Source)
    at org.gradle.api.internal.tasks.testing.worker.TestWorker$3.run(TestWorker.java:193)
    at org.gradle.api.internal.tasks.testing.worker.TestWorker.executeAndMaintainThreadName(TestWorker.java:129)
    at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:100)
    at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:60)
    at org.gradle.process.internal.worker.child.ActionExecutionWorker.execute(ActionExecutionWorker.java:56)
    at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:113)
    at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:65)
    at worker.org.gradle.process.internal.worker.GradleWorkerMain.run(GradleWorkerMain.java:69)
    at worker.org.gradle.process.internal.worker.GradleWorkerMain.main(GradleWorkerMain.java:74)

[jlink]

I just tried this successfully with the latest version of jqwik (openjdk 17):

interface EqualityContract<T> {
    @Property
    @Report(Reporting.GENERATED)
    default boolean reflexivityEq(@ForAll("anyT") T value) {
        return Objects.equals(value, value);
    }

    Arbitrary<T> anyT();
}

class MyTypeTest implements EqualityContract<MyType> {

    @Provide("anyT")
    public Arbitrary<MyType> anyT() {
        return Arbitraries.strings().alpha().map(MyType::new);
    }
}

class MyType {
    private final String name;

    MyType(String name) {
        this.name = name;
    }

    @Override
    public String toString() {
        return "My(" + name + ")";
    }
}

Maybe an old version is loaded somehow? Or Java version has an influence?

[SimY4]

@jlink Sorry, getting more details as I look into this problem more. My simplified example is not representable and in reality was a bit more nuanced:

interface EqualityContract<T> {
  @Property
  default boolean reflexivityEq(@ForAll("anyT") T value) {
    return Objects.equals(value, value);
  }
  Arbitrary<T> anyT();
}

interface ComparableContract<T extends Comparable<? super T>> extends EqualityContract<T> {}

class MyTypeTest implements ComparableContract<String> {
  @Provide("anyT")
  Arbitrary<String> anyT() { return Arbitraries.strings(); }
}

[jlink]

Still seems to work with the extended contract. I assume something else is going on.

[SimY4]

@jlink It's yet another weird one... If I try to debug it and stall an execution it'll pass for me too. But running a test in a class would make it fail 100% of time on my machine. This makes me wonder where this race could come from. 🤔

In my case, I see that CachingArbitraryResolver cached the value of an empty set for my type parameter. But if I stall it on compute if absent it'll return 5 arbitraries.

[jlink]

Maybe you can set up a repo that I can use to replicate the problem.

[SimY4]

Found minimal example that breaks:

interface A<T> {
  @Property
  default boolean test1(@ForAll("anyT") T t) {
    return true;
  }

  Arbitrary<T> anyT();
}

interface B<T extends Comparable<? super T>> extends A<T> {
  @Property
  default boolean test2(@ForAll("anyT") T t) {
    return true;
  }
}

interface MyType extends Comparable<MyType> { }

interface C<T extends MyType> extends B<T> {
  @Property
  default boolean test3(@ForAll("anyT") T t) {
    return true;
  }
}

class MyRecord implements MyType {
  @Override
  public int compareTo(MyType o) {
    return 0;
  }
}

public class Test implements C<MyRecord> {
  @Provide("anyT")
  @Override
  public Arbitrary<MyRecord> anyT() {
    return Arbitraries.of(new MyRecord());
  }
}

[jlink]

@SimY4 Have you seen this work in older versions of jqwik?

[SimY4]

@jlink Yeah, it works on 1.7.4 but it could be by chance. IDK...

[jlink]

@SimY4 Sorry for the delay. I can reproduce. Looks like a bug. Will try to dive into it in the week to come.

[SimY4]

@jlink no problem. Thank you for looking into this

[jlink]

Should be fixed in https://github.com/jqwik-team/jqwik/commit/d6de16dbb19814eaab0426bc2d08135ef84c864e

[jlink]

Fix released in 1.8.1-SNAPSHOT

[SimY4]

Awesome work! Thank you.🙏

[jlink]

@SimY4 Do you want to update to 1.8? In that case I could publish 1.8.1 soon.

[SimY4]

@jlink i did check the 1.8.1 snapshot on my project. Worked great, no problems detected, all tests are green. 😀

[jlink]

1.8.1 has been released