adding `jqwik-kotlin` causes existing tests written in java to fail with NPE

[twentylemon]

Testing Problem

I have a java code base that we recently starting adding some kotlin to. When adding jqwik-kotlin as a dependency, and seemingly doing the rest of the setup, our existing tests in java fail before even starting to run anything.

I've a minimal test written in java,

class SomeTest {
  @Property
  void test(@ForAll int i) {
    System.out.println(i);
  }
}

Snips of pom,

<kotlin.version>1.9.22</kotlin.version>
<jqwik.version>1.8.3</jqwik.version>
...

      <dependency>
        <groupId>net.jqwik</groupId>
        <artifactId>jqwik</artifactId>
        <version>${jqwik.version}</version>
        <scope>test</scope>
      </dependency>
      <dependency>
        <groupId>net.jqwik</groupId>
        <artifactId>jqwik-kotlin</artifactId>
        <version>${jqwik.version}</version>
        <scope>test</scope>
      </dependency>
...
      <plugin>
        <groupId>org.jetbrains.kotlin</groupId>
        <artifactId>kotlin-maven-plugin</artifactId>
        <version>${kotlin.version}</version>
        <extensions>true</extensions>
        <configuration>
          <compilerPlugins>
            <plugin>lombok</plugin>
          </compilerPlugins>
          <pluginOptions>
            <option>lombok:config=${project.basedir}/lombok.config</option>
          </pluginOptions>
          <args>
            <arg>-java-parameters</arg> <!-- Get correct parameter names in jqwik reporting -->
            <arg>-Xjsr305=strict</arg> <!-- Strict interpretation of nullability annotations in jqwik API -->
            <arg>-Xemit-jvm-type-annotations</arg> <!-- Enable annotations on type variables -->
          </args>
        </configuration>
...
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-compiler-plugin</artifactId>
        <version>3.5.1</version>
        <configuration>
          <compilerArgs>
            <arg>-parameters</arg> <!-- required if you want to report source code names of property method parameters -->
          </compilerArgs>
        </configuration>

Removing jqwik-kotlin and re-running the build, the tests start to pass again.

Stack trace I am getting,

java.lang.NullPointerException: Parameter specified as non-null is null: method kotlin.reflect.jvm.internal.impl.descriptors.runtime.structure.ReflectJavaAnnotation.<init>, parameter annotation

    at kotlin.reflect.jvm.internal.impl.descriptors.runtime.structure.ReflectJavaAnnotation.<init>(ReflectJavaAnnotation.kt)
    at kotlin.reflect.jvm.internal.impl.descriptors.runtime.structure.ReflectJavaAnnotationOwnerKt.getAnnotations(ReflectJavaAnnotationOwner.kt:37)
    at kotlin.reflect.jvm.internal.impl.descriptors.runtime.structure.ReflectJavaTypeParameter.getAnnotations(ReflectJavaTypeParameter.kt:27)
    at kotlin.reflect.jvm.internal.impl.descriptors.runtime.structure.ReflectJavaTypeParameter.getAnnotations(ReflectJavaTypeParameter.kt:23)
    at kotlin.reflect.jvm.internal.impl.load.java.lazy.LazyJavaAnnotations.iterator(LazyJavaAnnotations.kt:40)
    at kotlin.collections.CollectionsKt__MutableCollectionsKt.addAll(MutableCollections.kt:117)
    at kotlin.collections.CollectionsKt___CollectionsKt.plus(_Collections.kt:3250)
    at kotlin.reflect.jvm.internal.impl.load.java.typeEnhancement.AbstractSignatureParts.extractQualifiersFromAnnotations(AbstractSignatureParts.kt:90)
    at kotlin.reflect.jvm.internal.impl.load.java.typeEnhancement.AbstractSignatureParts.computeIndexedQualifiers(AbstractSignatureParts.kt:181)
    at kotlin.reflect.jvm.internal.impl.load.java.typeEnhancement.SignatureEnhancement.enhance(signatureEnhancement.kt:223)
    at kotlin.reflect.jvm.internal.impl.load.java.typeEnhancement.SignatureEnhancement.enhance$default(signatureEnhancement.kt:217)
    at kotlin.reflect.jvm.internal.impl.load.java.typeEnhancement.SignatureEnhancement.enhanceTypeParameterBounds(signatureEnhancement.kt:169)
    at kotlin.reflect.jvm.internal.impl.load.java.lazy.descriptors.LazyJavaTypeParameterDescriptor.processBoundsWithoutCycles(LazyJavaTypeParameterDescriptor.kt:68)
    at kotlin.reflect.jvm.internal.impl.descriptors.impl.AbstractTypeParameterDescriptor$TypeParameterTypeConstructor.processSupertypesWithoutCycles(AbstractTypeParameterDescriptor.java:220)
    at kotlin.reflect.jvm.internal.impl.types.AbstractTypeConstructor$supertypes$3.invoke(AbstractTypeConstructor.kt:107)
    at kotlin.reflect.jvm.internal.impl.types.AbstractTypeConstructor$supertypes$3.invoke(AbstractTypeConstructor.kt:77)
    at kotlin.reflect.jvm.internal.impl.storage.LockBasedStorageManager$5.doPostCompute(LockBasedStorageManager.java:235)
    at kotlin.reflect.jvm.internal.impl.storage.LockBasedStorageManager$LockBasedLazyValueWithPostCompute.postCompute(LockBasedStorageManager.java:491)
    at kotlin.reflect.jvm.internal.impl.storage.LockBasedStorageManager$LockBasedLazyValue.invoke(LockBasedStorageManager.java:412)
    at kotlin.reflect.jvm.internal.impl.storage.LockBasedStorageManager$LockBasedLazyValueWithPostCompute.invoke(LockBasedStorageManager.java:481)
    at kotlin.reflect.jvm.internal.impl.storage.LockBasedStorageManager$LockBasedNotNullLazyValueWithPostCompute.invoke(LockBasedStorageManager.java:512)
    at kotlin.reflect.jvm.internal.impl.types.AbstractTypeConstructor.getSupertypes(AbstractTypeConstructor.kt:27)
    at kotlin.reflect.jvm.internal.impl.descriptors.impl.AbstractTypeParameterDescriptor.getUpperBounds(AbstractTypeParameterDescriptor.java:121)
    at kotlin.reflect.jvm.internal.impl.types.checker.ClassicTypeSystemContext$DefaultImpls.getUpperBounds(ClassicTypeSystemContext.kt:282)
    at kotlin.reflect.jvm.internal.impl.types.checker.SimpleClassicTypeSystemContext.getUpperBounds(NewKotlinTypeChecker.kt:29)
    at kotlin.reflect.jvm.internal.impl.load.java.typeEnhancement.AbstractSignatureParts.getBoundsNullability(AbstractSignatureParts.kt:151)
    at kotlin.reflect.jvm.internal.impl.load.java.typeEnhancement.AbstractSignatureParts.extractQualifiersFromAnnotations(AbstractSignatureParts.kt:129)
    at kotlin.reflect.jvm.internal.impl.load.java.typeEnhancement.AbstractSignatureParts.computeIndexedQualifiers(AbstractSignatureParts.kt:181)
    at kotlin.reflect.jvm.internal.impl.load.java.typeEnhancement.SignatureEnhancement.enhance(signatureEnhancement.kt:223)
    at kotlin.reflect.jvm.internal.impl.load.java.typeEnhancement.SignatureEnhancement.enhance(signatureEnhancement.kt:214)
    at kotlin.reflect.jvm.internal.impl.load.java.typeEnhancement.SignatureEnhancement.enhance$default(signatureEnhancement.kt:204)
    at kotlin.reflect.jvm.internal.impl.load.java.typeEnhancement.SignatureEnhancement.enhanceSignature(signatureEnhancement.kt:125)
    at kotlin.reflect.jvm.internal.impl.load.java.typeEnhancement.SignatureEnhancement.enhanceSignatures(signatureEnhancement.kt:56)
    at kotlin.reflect.jvm.internal.impl.load.java.lazy.descriptors.LazyJavaScope$functions$1.invoke(LazyJavaScope.kt:125)
    at kotlin.reflect.jvm.internal.impl.load.java.lazy.descriptors.LazyJavaScope$functions$1.invoke(LazyJavaScope.kt:118)
    at kotlin.reflect.jvm.internal.impl.storage.LockBasedStorageManager$MapBasedMemoizedFunction.invoke(LockBasedStorageManager.java:578)
    at kotlin.reflect.jvm.internal.impl.storage.LockBasedStorageManager$MapBasedMemoizedFunctionToNotNull.invoke(LockBasedStorageManager.java:681)
    at kotlin.reflect.jvm.internal.impl.load.java.lazy.descriptors.LazyJavaScope.getContributedFunctions(LazyJavaScope.kt:273)
    at kotlin.reflect.jvm.internal.impl.load.java.lazy.descriptors.LazyJavaClassMemberScope.getContributedFunctions(LazyJavaClassMemberScope.kt:865)
    at kotlin.reflect.jvm.internal.impl.load.java.lazy.descriptors.LazyJavaScope.computeDescriptors(LazyJavaScope.kt:378)
    at kotlin.reflect.jvm.internal.impl.load.java.lazy.descriptors.LazyJavaScope$allDescriptors$1.invoke(LazyJavaScope.kt:64)
    at kotlin.reflect.jvm.internal.impl.load.java.lazy.descriptors.LazyJavaScope$allDescriptors$1.invoke(LazyJavaScope.kt:63)
    at kotlin.reflect.jvm.internal.impl.storage.LockBasedStorageManager$LockBasedLazyValue.invoke(LockBasedStorageManager.java:408)
    at kotlin.reflect.jvm.internal.impl.storage.LockBasedStorageManager$LockBasedNotNullLazyValue.invoke(LockBasedStorageManager.java:527)
    at kotlin.reflect.jvm.internal.impl.load.java.lazy.descriptors.LazyJavaScope.getContributedDescriptors(LazyJavaScope.kt:357)
    at kotlin.reflect.jvm.internal.impl.resolve.scopes.ResolutionScope$DefaultImpls.getContributedDescriptors$default(ResolutionScope.kt:50)
    at kotlin.reflect.jvm.internal.KDeclarationContainerImpl.getMembers(KDeclarationContainerImpl.kt:58)
    at kotlin.reflect.jvm.internal.KClassImpl$Data$declaredNonStaticMembers$2.invoke(KClassImpl.kt:173)
    at kotlin.reflect.jvm.internal.KClassImpl$Data$declaredNonStaticMembers$2.invoke(KClassImpl.kt:173)
    at kotlin.reflect.jvm.internal.ReflectProperties$LazySoftVal.invoke(ReflectProperties.java:70)
    at kotlin.reflect.jvm.internal.ReflectProperties$Val.getValue(ReflectProperties.java:32)
    at kotlin.reflect.jvm.internal.KClassImpl$Data.getDeclaredNonStaticMembers(KClassImpl.kt:173)
    at kotlin.reflect.jvm.internal.KClassImpl$Data$allNonStaticMembers$2.invoke(KClassImpl.kt:182)
    at kotlin.reflect.jvm.internal.KClassImpl$Data$allNonStaticMembers$2.invoke(KClassImpl.kt:182)
    at kotlin.reflect.jvm.internal.ReflectProperties$LazySoftVal.invoke(ReflectProperties.java:70)
    at kotlin.reflect.jvm.internal.ReflectProperties$Val.getValue(ReflectProperties.java:32)
    at kotlin.reflect.jvm.internal.KClassImpl$Data.getAllNonStaticMembers(KClassImpl.kt:182)
    at kotlin.reflect.jvm.internal.KClassImpl$Data$allMembers$2.invoke(KClassImpl.kt:188)
    at kotlin.reflect.jvm.internal.KClassImpl$Data$allMembers$2.invoke(KClassImpl.kt:188)
    at kotlin.reflect.jvm.internal.ReflectProperties$LazySoftVal.invoke(ReflectProperties.java:70)
    at kotlin.reflect.jvm.internal.ReflectProperties$Val.getValue(ReflectProperties.java:32)
    at kotlin.reflect.jvm.internal.KClassImpl$Data.getAllMembers(KClassImpl.kt:188)
    at kotlin.reflect.jvm.internal.KClassImpl.getMembers(KClassImpl.kt:206)
    at kotlin.reflect.full.KClasses.getFunctions(KClasses.kt:89)
    at kotlin.reflect.jvm.ReflectJvmMapping.getKotlinFunction(ReflectJvmMapping.kt:144)
    at net.jqwik.kotlin.internal.SuspendedPropertyMethodsHook.isSuspendFunction(SuspendedPropertyMethodsHook.kt:41)
    at net.jqwik.kotlin.internal.SuspendedPropertyMethodsHook.access$isSuspendFunction(SuspendedPropertyMethodsHook.kt:19)
    at net.jqwik.kotlin.internal.SuspendedPropertyMethodsHook$appliesTo$1.invoke(SuspendedPropertyMethodsHook.kt:24)
    at net.jqwik.kotlin.internal.SuspendedPropertyMethodsHook$appliesTo$1.invoke(SuspendedPropertyMethodsHook.kt:24)
    at net.jqwik.kotlin.internal.SuspendedPropertyMethodsHook.appliesTo$lambda$0(SuspendedPropertyMethodsHook.kt:24)
    at java.base/java.util.Optional.map(Optional.java:260)
    at net.jqwik.kotlin.internal.SuspendedPropertyMethodsHook.appliesTo(SuspendedPropertyMethodsHook.kt:24)
    at net.jqwik.kotlin.internal.SuspendedPropertyMethodsHook.appliesTo(SuspendedPropertyMethodsHook.kt:19)
    at net.jqwik.engine.execution.lifecycle.LifecycleHooksRegistry.hookAppliesTo(LifecycleHooksRegistry.java:109)
    at net.jqwik.engine.execution.lifecycle.LifecycleHooksRegistry.lambda$findHooks$1(LifecycleHooksRegistry.java:102)
    at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:178)
    at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
    at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625)
    at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
    at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
    at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921)
    at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
    at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682)
    at net.jqwik.engine.execution.lifecycle.LifecycleHooksRegistry.findHooks(LifecycleHooksRegistry.java:104)
    at net.jqwik.engine.execution.lifecycle.LifecycleHooksRegistry.resolveParameterHook(LifecycleHooksRegistry.java:58)
    at net.jqwik.engine.execution.PropertyTaskCreator.createLifecycleContext(PropertyTaskCreator.java:87)
    at net.jqwik.engine.execution.PropertyTaskCreator.lambda$createTask$2(PropertyTaskCreator.java:34)
    at net.jqwik.engine.execution.pipeline.ExecutionTask$1.lambda$execute$0(ExecutionTask.java:31)
    at net.jqwik.engine.execution.lifecycle.CurrentTestDescriptor.runWithDescriptor(CurrentTestDescriptor.java:17)
    at net.jqwik.engine.execution.pipeline.ExecutionTask$1.execute(ExecutionTask.java:31)
    at net.jqwik.engine.execution.pipeline.ExecutionPipeline.runToTermination(ExecutionPipeline.java:82)
    at net.jqwik.engine.execution.JqwikExecutor.execute(JqwikExecutor.java:46)
    at net.jqwik.engine.JqwikTestEngine.executeTests(JqwikTestEngine.java:70)
    at net.jqwik.engine.JqwikTestEngine.execute(JqwikTestEngine.java:53)
    at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:147)
    at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:127)
    at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:90)
    at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:55)
    at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:102)
    at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:54)
    at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:114)
    at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:86)
    at org.junit.platform.launcher.core.DefaultLauncherSession$DelegatingLauncher.execute(DefaultLauncherSession.java:86)
    at org.junit.platform.launcher.core.SessionPerRequestLauncher.execute(SessionPerRequestLauncher.java:53)
    at com.intellij.junit5.JUnit5IdeaTestRunner.startRunnerWithArgs(JUnit5IdeaTestRunner.java:57)
    at com.intellij.rt.junit.IdeaTestRunner$Repeater$1.execute(IdeaTestRunner.java:38)
    at com.intellij.rt.execution.junit.TestsRepeater.repeat(TestsRepeater.java:11)
    at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:35)
    at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:232)
    at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:55)

That's from intellij, but the trace is essentially the same as running from mvn test, just the root changes to surefire.

Suggested Solution

Discussion

[jlink]

@twentylemon Thanks for reporting this issue. Have you tried older versions of jqwik and/or Kotlin? Have you tried without Lombok being present?

[twentylemon]

I have tried with jqwik 1.8.1, I upgraded as part of debugging. The project contains a lot of Lombok, so removing that isn't a realistic option -- maybe in a few months once most code becomes kotlin.

About Lombok, there isn't any used in the test class. Makes me feel like it shouldn't be an issue. But Lombok has definitely surprised me many times before.

I can try downgrading kotlin, but I am pinned to 1.9. It's a work project, so I'll report back after the weekend 🙂

[jlink]

Do you have an isolated repo to replicate the problem?

I cannot replicate your problem. I duplicated your setup except lombok being enabled since that seems to require a config file that I haven't.

Maybe the java version plays a role. My properties look like that:

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <maven.compiler.source>1.8</maven.compiler.source>
        <maven.compiler.target>${maven.compiler.source}</maven.compiler.target>
        <jqwik.version>1.8.3</jqwik.version>
        <kotlin.version>1.9.22</kotlin.version>
    </properties>

[jlink]

But something seems to be wrong with the kotlin-maven-plugin. At least on my machine, IntelliJ no longer marks the Kotlin test classes and methods with the "Run Test" triangles; that's never a problem in my Kotlin Gradle set ups.

[twentylemon]

But something seems to be wrong with the kotlin-maven-plugin. At least on my machine, IntelliJ no longer marks the Kotlin test classes and methods with the "Run Test" triangles; that's never a problem in my Kotlin Gradle set ups.

I haven't noticed anything like that. I've used the hotkey for run tests many times without fail. The kotlin-lombok plugin is experimental, but the overall kotlin-maven one -- at least I haven't noticed anything unusual.

I duplicated your setup except lombok being enabled since that seems to require a config file that I haven't.

IIRC that file is effectively empty. I can confirm after the weekend though.

Do you have an isolated repo to replicate the problem?

I don't. I'll try to whip one up; again, after the weekend though.

Thanks for quick responses!

[twentylemon]

@jlink I was able to get a minimal repo together: https://github.com/twentylemon/jqwik-kotlin-npe -- lombok seems to be a red herring, I was able to remove that altogether and still see the error.

I tried downgraded kotlin to 1.8.0, but then my tests down run at all :thinking:

[jlink]

@twentylemon I can replicate! Thanks.

[jlink]

@twentylemon Looks like a bug in Kotlin library. I'll probably have to find a workaround.

[jlink]

@twentylemon Please Try release "1.8.4-SNAPSHOT"

[twentylemon]

Seems my workplace doesn't have snapshot releases. They lock down maven central for """security reasons.""" I'll be able to try later tonight on my personal machine.

[twentylemon]

Using 1.8.4 in my example repo worked! Unfortunately I wasn't able to test it out on my work machine in a larger repo, though I think I realize now the issue was the wrong snapshots url. I didn't realize you used s01.oss.yadayadayada. I'll try again at work tomorrow, but it does seem like it should be solved 😄

[jlink]

s01.oss

It's a faster instance than the default one. I switched over a year ago but probably forgot to update some references in the jqwik-samples repository.

[twentylemon]

Even with the correct snapshots repo, seems I am still blocked on my work pc. Oddly though, I can just download the damn jar, so that makes sense.

After adding that as a system dependency, tests are indeed running normally on the big repo as well 🎉

[jlink]

@twentylemon I'll mark this as closed. Feel free to reopen if some problem with the current solution shows up later.

[twentylemon]

@jlink Would this fix be part of the 1.8.4 release? Any clue when that would be available? I'd like to use the library for the big project, but I can't rely on a static jar or a snapshot release for that.

[jlink]

I could release it at the weekend if that helps you.

[twentylemon]

I don't mean to rush you. Whatever your release schedule normally is, that's good by me. Would just like to know so I can start using it 😄

[jlink]

1.8.4 has just been released.