ascopes / java-compiler-testing

Write sandboxed integration tests for Java annotation processors and plugins.
https://ascopes.github.io/java-compiler-testing/
Apache License 2.0
13 stars 10 forks source link

[BUG] You must register at least one TestTemplateInvocationContextProvider that supports @TestTemplate #499

Closed natros closed 1 year ago

natros commented 1 year ago
import static io.github.ascopes.jct.assertions.JctAssertions.assertThatCompilation;

import io.github.ascopes.jct.compilers.JctCompiler;
import io.github.ascopes.jct.junit.JavacCompilerTest;
import io.github.ascopes.jct.junit.JctExtension;
import io.github.ascopes.jct.junit.Managed;
import io.github.ascopes.jct.workspaces.Workspace;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.extension.ExtendWith;

@DisplayName("Example tests")
@ExtendWith(JctExtension.class)
class ExampleTest {
  @Managed Workspace workspace;

  @DisplayName("I can compile a Hello World application")
  @JavacCompilerTest
  void canCompileHelloWorld(JctCompiler compiler) {
    // Given
    workspace
        .createSourcePathPackage()
        .createFile("org/example/Message.java")
        .withContents(
            """
            package org.example;

            import lombok.Data;
            import lombok.NonNull;

            @Data
            public class Message {
              private String content;

              public static void main(String[] args) {
                Message message = new Message("Hello, World!");
                System.out.println(message);
              }
            }
            """);

    // When
    var compilation = compiler.compile(workspace);

    // Then
    assertThatCompilation(compilation).isSuccessfulWithoutWarnings();

    assertThatCompilation(compilation)
        .classOutputPackages()
        .fileExists("com/example/Message.class")
        .isNotEmptyFile();
  }
}
$ java --version
openjdk 17.0.7 2023-04-18 LTS
OpenJDK Runtime Environment Zulu17.42+19-CA (build 17.0.7+7-LTS)
OpenJDK 64-Bit Server VM Zulu17.42+19-CA (build 17.0.7+7-LTS, mixed mode, sharing)

Junit 5.9.3

You must register at least one TestTemplateInvocationContextProvider that supports @TestTemplate method [void com.github.natros.wf.resteasy.processor.ExampleTest.canCompileHelloWorld(io.github.ascopes.jct.compilers.JctCompiler)]
org.junit.platform.commons.PreconditionViolationException: You must register at least one TestTemplateInvocationContextProvider that supports @TestTemplate method [void com.github.natros.wf.resteasy.processor.ExampleTest.canCompileHelloWorld(io.github.ascopes.jct.compilers.JctCompiler)]
    at org.junit.platform.commons.util.Preconditions.condition(Preconditions.java:314)
    at org.junit.platform.commons.util.Preconditions.notEmpty(Preconditions.java:164)
    at org.junit.jupiter.engine.descriptor.TestTemplateTestDescriptor.validateProviders(TestTemplateTestDescriptor.java:125)
    at org.junit.jupiter.engine.descriptor.TestTemplateTestDescriptor.execute(TestTemplateTestDescriptor.java:101)
    at org.junit.jupiter.engine.descriptor.TestTemplateTestDescriptor.execute(TestTemplateTestDescriptor.java:44)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:151)
    at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
    at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
    at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
    at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
    at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155)
    at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
    at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
    at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
    at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
    at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155)
    at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
    at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
    at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
    at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
    at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:35)
    at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
    at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:54)
    at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:107)
    at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:88)
    at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:54)
    at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:67)
    at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:52)
    at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:114)
    at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:86)
    at org.junit.platform.launcher.core.DefaultLauncherSession$DelegatingLauncher.execute(DefaultLauncherSession.java:86)
    at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.processAllTestClasses(JUnitPlatformTestClassProcessor.java:110)
    at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.access$000(JUnitPlatformTestClassProcessor.java:90)
    at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor.stop(JUnitPlatformTestClassProcessor.java:85)
    at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.stop(SuiteTestClassProcessor.java:62)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:568)
    at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
    at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
    at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
    at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
    at jdk.proxy1/jdk.proxy1.$Proxy2.stop(Unknown Source)
    at org.gradle.api.internal.tasks.testing.worker.TestWorker$3.run(TestWorker.java:193)
    at org.gradle.api.internal.tasks.testing.worker.TestWorker.executeAndMaintainThreadName(TestWorker.java:129)
    at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:100)
    at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:60)
    at org.gradle.process.internal.worker.child.ActionExecutionWorker.execute(ActionExecutionWorker.java:56)
    at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:113)
    at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:65)
    at worker.org.gradle.process.internal.worker.GradleWorkerMain.run(GradleWorkerMain.java:69)
    at worker.org.gradle.process.internal.worker.GradleWorkerMain.main(GradleWorkerMain.java:74)
ascopes commented 1 year ago

Thanks for this. I'll take a look a bit later and try getting a fix out today at some point. Do you happen to have the Gradle build script you were using, just to help me replicate this?

natros commented 1 year ago

jct.zip

ascopes commented 1 year ago

@natros thanks. Got it running on my PC and I can reproduce the issue too.

Really strange that it is not occurring during the build process for this repository.

I haven't got much experience using Gradle. I can use it if I need to but I tend to prefer Maven just because I have more exposure to it. Do you happen to know if Gradle does anything significantly different to maven-surefire as part of their test invocation process? If you don't, that is fine, I can have a look into it.

I think the issue is possibly just that we don't need to use @TestTemplate though, so I'll have a fiddle around with this locally and try to find a fix. I'll drop a response once I get somewhere with it!

As a side note, I did notice a little mistake in the snippet you are trying to compile. It looks like I made this mistake when writing the README (which I updated a little earlier today to be correct). The correct test case you'll want to use is:

package demo;

import static io.github.ascopes.jct.assertions.JctAssertions.assertThatCompilation;

import io.github.ascopes.jct.compilers.JctCompiler;
import io.github.ascopes.jct.junit.JavacCompilerTest;
import io.github.ascopes.jct.junit.JctExtension;
import io.github.ascopes.jct.junit.Managed;
import io.github.ascopes.jct.workspaces.Workspace;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.extension.ExtendWith;

@DisplayName("Example tests")
@ExtendWith(JctExtension.class)
class ExampleTest {
  @Managed Workspace workspace;

  @DisplayName("I can compile a Hello World application")
  @JavacCompilerTest
  void canCompileHelloWorld(JctCompiler compiler) {
    // Given
    workspace
        .createSourcePathPackage()
        .createFile("org/example/Message.java")
        .withContents(
            """
            package org.example;

            import lombok.Data;
            import lombok.NonNull;

            @Data
            public class Message {
              private final String content;

              public static void main(String[] args) {
                Message message = new Message("Hello, World!");
                System.out.println(message);
              }
            }
            """);

    // When
    var compilation = compiler.compile(workspace);

    // Then
    assertThatCompilation(compilation).isSuccessfulWithoutWarnings();

    assertThatCompilation(compilation)
        .classOutputPackages()
        .fileExists("com/example/Message.class")
        .isNotEmptyFile();
  }
}

The difference is just that the content field needs to be marked as final otherwise Lombok will not enforce that it is part of a constructor.

ascopes commented 1 year ago

Think this is going to take a bit more work. It looks like Gradle is not handling the meta-annotations for @ParameterizedTest in the same way that Maven is doing in Maven Surefire, which is a fairly big pain as it means I need to implement parameterized test resolution from scratch.

My (semi-unfortunate) assumption was that Gradle would make use of the same APIs that Maven did for invocations of JUnit tests.

I don't think I am going to be able to get this working today, but I will try to get it done as soon as possible. I am working all week this week and then I am away for a week so I won't have access to a PC, but I will put this at the top of my priority list.

For now, the workaround is to avoid the JUnit integration and use the JctCompilers.newPlatformCompiler API directly.

Thanks for taking the time to report this :-)


If this is the case, it will involve:

natros commented 1 year ago

there is a missing dependency in gradle testImplementation("org.junit.jupiter:junit-jupiter-params:5.9.3")

that in your pom file is optional

<dependency>
      <!-- Used to provide convenience annotations -->
      <groupId>org.junit.jupiter</groupId>
      <artifactId>junit-jupiter-params</artifactId>
      <optional>true</optional>
    </dependency>

there is no need for big changes

ascopes commented 1 year ago

Oh, interesting, I totally missed that!

Does that fix your issue on your side?

If it does, do you want to submit a fix from your side? If not, I can take a look at updating it from my side when I am next at a PC.

natros commented 1 year ago

It works, but I had to make a few adjustments. 👍

ascopes commented 1 year ago

Thanks. Lombok here was just an example of how it pulls through existing annotation processors on the classpath. But yeah, that package name definitely needs fixing. Good spot

natros commented 1 year ago

I understand the intent. But the first thing anyone does is copy&paste the example to see it working. It's frustrating when that doesn't happen.

ascopes commented 1 year ago

Will update the example. Thanks for the feedback!

ascopes commented 1 year ago

I forgot to mention this last week as I had other non-work related stuff going on, but if you have any additional feedback or suggestions, please feel free to open a discussion or ticket!