microsoft / vscode-java-test

Run and debug Java test cases in Visual Studio Code.
https://marketplace.visualstudio.com/items?itemName=vscjava.vscode-java-test
Other
292 stars 125 forks source link

Micronaut 4 Gradle project test execution broken #1613

Closed dbalek closed 2 months ago

dbalek commented 11 months ago

Generate a sample Micronaut project using Micronaut Launch service with Micronaut 4.1.1, Java 17, Gradle, and JUnit selected. Open generated project in VSCode with the Extension Pack for Java installed. Try to run project tests either via Test Explorer or by clicking Run Test icons in source editor gutter. No test gets executed and the following error appear in the console:

org.junit.platform.commons.JUnitException: TestEngine with ID 'junit-jupiter' failed to discover tests
    at org.junit.platform.launcher.core.EngineDiscoveryOrchestrator.discoverEngineRoot(EngineDiscoveryOrchestrator.java:160)
EngineDiscoveryOrchestrator.java:160
    at org.junit.platform.launcher.core.EngineDiscoveryOrchestrator.discoverSafely(EngineDiscoveryOrchestrator.java:132)
EngineDiscoveryOrchestrator.java:132
    at org.junit.platform.launcher.core.EngineDiscoveryOrchestrator.discover(EngineDiscoveryOrchestrator.java:107)
EngineDiscoveryOrchestrator.java:107
    at org.junit.platform.launcher.core.EngineDiscoveryOrchestrator.discover(EngineDiscoveryOrchestrator.java:78)
EngineDiscoveryOrchestrator.java:78
    at org.junit.platform.launcher.core.DefaultLauncher.discover(DefaultLauncher.java:99)
DefaultLauncher.java:99
    at org.junit.platform.launcher.core.DefaultLauncher.discover(DefaultLauncher.java:77)
DefaultLauncher.java:77
    at org.junit.platform.launcher.core.DelegatingLauncher.discover(DelegatingLauncher.java:42)
DelegatingLauncher.java:42
    at org.junit.platform.launcher.core.SessionPerRequestLauncher.discover(SessionPerRequestLauncher.java:56)
SessionPerRequestLauncher.java:56
    at org.eclipse.jdt.internal.junit5.runner.JUnit5TestReference.<init>(JUnit5TestReference.java:46)
    at org.eclipse.jdt.internal.junit5.runner.JUnit5TestLoader.createUnfilteredTest(JUnit5TestLoader.java:88)
    at org.eclipse.jdt.internal.junit5.runner.JUnit5TestLoader.createTest(JUnit5TestLoader.java:69)
    at org.eclipse.jdt.internal.junit5.runner.JUnit5TestLoader.loadTests(JUnit5TestLoader.java:56)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:513)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:756)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:452)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:210)
Caused by: org.junit.platform.commons.JUnitException: ClassSelector [className = 'com.example.Mn4gradleTest', classLoader = jdk.internal.loader.ClassLoaders$AppClassLoader@251a69d7] resolution failed
    at org.junit.platform.launcher.listeners.discovery.AbortOnFailureLauncherDiscoveryListener.selectorProcessed(AbortOnFailureLauncherDiscoveryListener.java:39)
AbortOnFailureLauncherDiscoveryListener.java:39
    at org.junit.platform.engine.support.discovery.EngineDiscoveryRequestResolution.resolveCompletely(EngineDiscoveryRequestResolution.java:103)
EngineDiscoveryRequestResolution.java:103
    at org.junit.platform.engine.support.discovery.EngineDiscoveryRequestResolution.run(EngineDiscoveryRequestResolution.java:83)
EngineDiscoveryRequestResolution.java:83
    at org.junit.platform.engine.support.discovery.EngineDiscoveryRequestResolver.resolve(EngineDiscoveryRequestResolver.java:113)
EngineDiscoveryRequestResolver.java:113
    at org.junit.jupiter.engine.discovery.DiscoverySelectorResolver.resolveSelectors(DiscoverySelectorResolver.java:46)
DiscoverySelectorResolver.java:46
    at org.junit.jupiter.engine.JupiterTestEngine.discover(JupiterTestEngine.java:69)
JupiterTestEngine.java:69
    at org.junit.platform.launcher.core.EngineDiscoveryOrchestrator.discoverEngineRoot(EngineDiscoveryOrchestrator.java:152)
EngineDiscoveryOrchestrator.java:152
    ... 15 more
Caused by: java.lang.NoSuchMethodError: 'java.util.stream.Stream org.junit.platform.commons.support.ReflectionSupport.streamNestedClasses(java.lang.Class, java.util.function.Predicate)'
    at org.junit.jupiter.engine.discovery.ClassSelectorResolver.lambda$toResolution$12(ClassSelectorResolver.java:138)
ClassSelectorResolver.java:138
    at org.junit.platform.engine.support.discovery.SelectorResolver$Match.expand(SelectorResolver.java:668)
SelectorResolver.java:668
    at org.junit.platform.engine.support.discovery.EngineDiscoveryRequestResolution.lambda$enqueueAdditionalSelectors$1(EngineDiscoveryRequestResolution.java:110)
EngineDiscoveryRequestResolution.java:110
    at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
ForEachOps.java:183
    at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:179)
ReferencePipeline.java:179
    at java.base/java.util.Collections$2.tryAdvance(Collections.java:4853)
Collections.java:4853
    at java.base/java.util.Collections$2.forEachRemaining(Collections.java:4861)
Collections.java:4861
    at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
AbstractPipeline.java:509
    at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
AbstractPipeline.java:499
    at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
ForEachOps.java:150
    at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
ForEachOps.java:173
    at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
AbstractPipeline.java:234
    at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
ReferencePipeline.java:596
    at org.junit.platform.engine.support.discovery.EngineDiscoveryRequestResolution.enqueueAdditionalSelectors(EngineDiscoveryRequestResolution.java:109)
EngineDiscoveryRequestResolution.java:109
    at org.junit.platform.engine.support.discovery.EngineDiscoveryRequestResolution.resolveCompletely(EngineDiscoveryRequestResolution.java:95)
EngineDiscoveryRequestResolution.java:95
    ... 20 more
jdneo commented 11 months ago

It's probably caused by some API changes between JUnit 5.9 & 5.10.

I was trying to update the JDT Test to the latest version but unfortunately it will cause the legacy JUnit tests fails. See: https://github.com/microsoft/vscode-java-test/pull/1608

graemerocher commented 11 months ago

Why is the IDE embedding a version of JUnit that is incompatible with the version used by the project? This seems poorly thought out as these versions will unlikely align

jdneo commented 11 months ago

I need to correct my statement before; the problem is not caused by the embedded JUnit libraries.

The test runner will use the project's JUnit dependencies if they are available. Only when the required dependencies are not declared by the project, the test runner will use the embedded ones.

The problem is caused due to some concept mismatch between the JDT project and the Gradle project.

If I run ./gradlew dependencies from the sample project, I can see both JUnit 5.9.3 and 5.10.0 are declared in different source sets. That is fine for a Gradle project, because in Gradle projects, each source set can have its own classpath.

Unfortunately, the concept of source set does not exist for a JDT project. One JDT project can only have one .classpath file which persists the classpath of the whole project. So, when importing a Gradle project to a JDT project, a 'workaround' is done to mitigate this kind of mismatch, that is to merge the classpath from all Gradle source sets into one.

For the sample project, the problem happens after merging. Both 5.9.3 and 5.10.0 are used in the classpath. Then who comes first wins -- the mismatch happens!

From the JDT side, there are two options to solve the issue:

Maybe, from Micronaut side, a workaround is to align the JUnit version for all the sourceset. But I admit that, to entirely solve the issue, something needs to be done at JDT side.

graemerocher commented 11 months ago

so if you run:

/gradlew dependencies --configuration runtimeClasspath | grep junit    
|    +--- org.junit:junit-bom:5.9.3
|    |    +--- org.junit:junit-bom:5.9.3

The only thing that is there is a BOM which should not be in the classpath. if you run:

./gradlew dependencies --configuration testRuntimeClasspath | grep junit
|    +--- org.junit:junit-bom:5.9.3 -> 5.10.0
|    |    +--- org.junit.jupiter:junit-jupiter:5.10.0 (c)
|    |    +--- org.junit.jupiter:junit-jupiter-api:5.10.0 (c)
|    |    +--- org.junit.jupiter:junit-jupiter-engine:5.10.0 (c)
|    |    +--- org.junit.platform:junit-platform-console:1.10.0 (c)
|    |    +--- org.junit.platform:junit-platform-launcher:1.10.0 (c)
|    |    +--- org.junit.jupiter:junit-jupiter-params:5.10.0 (c)
|    |    +--- org.junit.platform:junit-platform-commons:1.10.0 (c)
|    |    +--- org.junit.platform:junit-platform-engine:1.10.0 (c)
|    |    \--- org.junit.platform:junit-platform-reporting:1.10.0 (c)
|    |    +--- org.junit:junit-bom:5.9.3 -> 5.10.0 (*)
|    |    +--- io.micronaut.test:micronaut-test-junit5:4.0.2 (c)
+--- org.junit.jupiter:junit-jupiter-api -> 5.10.0
|    +--- org.junit:junit-bom:5.10.0 (*)
|    \--- org.junit.platform:junit-platform-commons:1.10.0
|         \--- org.junit:junit-bom:5.10.0 (*)
+--- io.micronaut.test:micronaut-test-junit5 -> 4.0.2
|    +--- org.junit.jupiter:junit-jupiter-api:5.9.3 -> 5.10.0 (*)
+--- org.graalvm.buildtools:junit-platform-native:0.9.25
|    +--- org.junit:junit-bom:5.10.0 (*)
|    +--- org.junit.platform:junit-platform-console:1.9.3 -> 1.10.0
|    |    +--- org.junit:junit-bom:5.10.0 (*)
|    |    \--- org.junit.platform:junit-platform-reporting:1.10.0
|    |         +--- org.junit:junit-bom:5.10.0 (*)
|    |         \--- org.junit.platform:junit-platform-launcher:1.10.0
|    |              +--- org.junit:junit-bom:5.10.0 (*)
|    |              \--- org.junit.platform:junit-platform-engine:1.10.0
|    |                   +--- org.junit:junit-bom:5.10.0 (*)
|    |                   \--- org.junit.platform:junit-platform-commons:1.10.0 (*)
|    +--- org.junit.platform:junit-platform-launcher:1.9.3 -> 1.10.0 (*)
|    \--- org.junit.jupiter:junit-jupiter -> 5.10.0
|         +--- org.junit:junit-bom:5.10.0 (*)
|         +--- org.junit.jupiter:junit-jupiter-api:5.10.0 (*)
|         +--- org.junit.jupiter:junit-jupiter-params:5.10.0
|         |    +--- org.junit:junit-bom:5.10.0 (*)
|         |    \--- org.junit.jupiter:junit-jupiter-api:5.10.0 (*)
|         \--- org.junit.jupiter:junit-jupiter-engine:5.10.0
|              +--- org.junit:junit-bom:5.10.0 (*)
|              +--- org.junit.platform:junit-platform-engine:1.10.0 (*)
|              \--- org.junit.jupiter:junit-jupiter-api:5.10.0 (*)
\--- org.junit.jupiter:junit-jupiter-engine -> 5.10.0 (*)

All the JUnit dependencies are promoted from 5.9.3 to 5.10.0 (the correct version). So it seems to me that the classpath produced by the JDT tooling is simply incorrect and the collection algorithm for dependencies incorrect and inconsistent with the behaviour of Gradle which is a massive problem

jdneo commented 11 months ago

The problem happens due to org.junit.platform:junit-platform-commons

+--- io.micronaut.platform:micronaut-platform:4.1.2
|    +--- org.junit:junit-bom:5.9.3
|    |    +--- org.junit.jupiter:junit-jupiter-api:5.9.3 (c)
|    |    \--- org.junit.platform:junit-platform-commons:1.9.3 (c)
Caused by: java.lang.NoSuchMethodError: 'java.util.stream.Stream org.junit.platform.commons.support.ReflectionSupport.streamNestedClasses(java.lang.Class, java.util.function.Predicate)'
graemerocher commented 11 months ago

if you look at the graph the correct version junit-platform-commons in my aforementioned comment is selected by the Gradle dependency configuration hence why it works in Gradle and not in VSCode

The bug appears to be that VScode is erroneously including the dependencies of io.micronaut.platform:micronaut-platform:4.1.2 which is a BOM not a dependency.

The classpath computation is clearly broken here.

We can of course "fix" this on the Micronaut side but the bug will still be there waiting to be hit again

graemerocher commented 11 months ago

A workaround is to add testImplementation("org.junit.platform:junit-platform-commons:1.10.0") to the Gradle build but this is a workaround and should never be necessary in the first place since VSCode/JDT is computing an erroneous classpath that differs from the one Gradle computes

graemerocher commented 11 months ago

Seems this is related to https://github.com/microsoft/vscode-java-test/issues/1020 which again sounds like the classpath computation for Gradle is simply incorrect

jdneo commented 11 months ago

I want to clarify that I'm not denying that current JDT cannot handle the problem perfectly. As I already said above:

But I admit that, to entirely solve the issue, something needs to be done at JDT side.

All the above comment I left is the analysis that I found why the classpath is wrong in JDT, and provide a possible workaround if it's still blocking before the issue is solved from JDT side. Please don't get me wrong.

MahatmaFatalError commented 8 months ago

I have the same problem with a Spring Boot gradle project in Eclipse. Using org.junit:junit-bom:5.10.1 that lead to org.junit.platform:junit-platform-commons:1.10.1

testImplementation("org.junit.platform:junit-platform-commons:1.10.0") fixed the problem, but I am puzzled why...

jdneo commented 2 months ago

Dup with https://github.com/microsoft/vscode-java-test/issues/1045#issuecomment-676911725.

There will be some update around this.

See: https://github.com/microsoft/build-server-for-gradle/issues/119

jdneo commented 1 month ago

The Gradle Test Delegation (both run and debug) has supported now.

To use this feature, you need to install the latest Test Runner for Java and Gradle for Java extension.

To delegate the tests to Gradle, you can set the default testing profile in Testing explorer: image image

If you do not want to change the default testing profile, you can trigger an one-time execution via: image image