search

junit-team / junit4

A programmer-oriented testing framework for Java.
https://junit.org/junit4
Eclipse Public License 1.0
8.53k stars 3.29k forks source link

Patch release 4.13.1 seems to break maven surefire plugin #1678

Closed honnix closed 4 years ago

honnix commented 4 years ago

Not super sure of this but I can describe the issue.

After bumping up to 4.13.1. our CI reports failing to parse "4.13.1:

Oct 16, 2020 10:04:34 AM org.junit.platform.launcher.core.DefaultLauncher handleThrowable
WARNING: TestEngine with ID 'junit-vintage' failed to discover tests
org.junit.platform.commons.JUnitException: Failed to parse version of junit:junit: 4.13.1
    at org.junit.vintage.engine.JUnit4VersionCheck.parseVersion(JUnit4VersionCheck.java:54)
    at org.junit.vintage.engine.JUnit4VersionCheck.checkSupported(JUnit4VersionCheck.java:37)
    at org.junit.vintage.engine.JUnit4VersionCheck.checkSupported(JUnit4VersionCheck.java:32)
    at org.junit.vintage.engine.VintageTestEngine.discover(VintageTestEngine.java:61)
    at org.junit.platform.launcher.core.DefaultLauncher.discoverEngineRoot(DefaultLauncher.java:168)
    at org.junit.platform.launcher.core.DefaultLauncher.discoverRoot(DefaultLauncher.java:155)
    at org.junit.platform.launcher.core.DefaultLauncher.discover(DefaultLauncher.java:120)
    at org.apache.maven.surefire.junitplatform.TestPlanScannerFilter.accept(TestPlanScannerFilter.java:56)
    at org.apache.maven.surefire.util.DefaultScanResult.applyFilter(DefaultScanResult.java:102)
    at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.scanClasspath(JUnitPlatformProvider.java:143)
    at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.getSuites(JUnitPlatformProvider.java:101)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:169)
    at org.apache.maven.surefire.util.ReflectionUtils.invokeGetter(ReflectionUtils.java:76)
    at org.apache.maven.surefire.util.ReflectionUtils.invokeGetter(ReflectionUtils.java:70)
    at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.getSuites(ProviderFactory.java:145)
    at org.apache.maven.plugin.surefire.booterclient.ForkStarter.getSuitesIterator(ForkStarter.java:698)
    at org.apache.maven.plugin.surefire.booterclient.ForkStarter.runSuitesForkPerTestSet(ForkStarter.java:418)
    at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:299)
    at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:247)
    at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeProvider(AbstractSurefireMojo.java:1161)
    at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeAfterPreconditionsChecked(AbstractSurefireMojo.java:1002)
    at org.apache.maven.plugin.surefire.AbstractSurefireMojo.execute(AbstractSurefireMojo.java:848)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:137)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:210)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:156)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:148)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute(MavenCli.java:956)
    at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
    at org.apache.maven.cli.MavenCli.main(MavenCli.java:192)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)

After looking into JUnit4VersionCheck, it seems it recently starts to support better versioning in this https://github.com/junit-team/junit5/commit/f30c96a9cad89a62a28750c5fe4dd83ad4333e99 commit that went to junit-vintage-engine:5.7.0`.

However the latest maven surefire plugins depends on junit-platform-launcher:1.3.2 that eventually depends on junit-vintage-engine:5.3.2 that doesn't support 4.13.1 as versioning scheme.

Forcing an upgrade:

        <plugin>
          <groupId>org.apache.maven.plugins</groupId>
          <artifactId>maven-surefire-plugin</artifactId>
          <version>3.0.0-M5</version>
          <dependencies>
            <dependency>
              <groupId>org.junit.platform</groupId>
              <artifactId>junit-platform-launcher</artifactId>
              <version>1.7.0</version>
            </dependency>
          </dependencies>
        </plugin>

breaks due to incompatible change over years:

org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M5:test failed: java.lang.NoClassDefFoundError: org/junit/platform/commons/util/ClassNamePatternFilterUtils: org.junit.platform.commons.util.ClassNamePatternFilterUtil

Maybe this issue fits better into maven surefire plugin repo?

sbrannen commented 4 years ago

Maybe this issue fits better into maven surefire plugin repo?

Yes, I agree with that.

@sormuras, do you have any further input?

marcphilipp commented 4 years ago

This was indeed fixed in https://github.com/junit-team/junit5/issues/2198 for JUnit Vintage Engine 5.7.

Can you upgrade to a 3.x version of Surefire?

honnix commented 4 years ago

@marcphilipp I couldn't find any new version than 3.0.0-M5 though.

marcphilipp commented 4 years ago

Ah, sorry I overlooked that part in the issue description. Since 4.13.1 works fine in our sample project I suspect you may have mixed versions of JUnit artifacts on the classpath. Could you please provide a sample project that reproduces the issue?

honnix commented 4 years ago

@marcphilipp I think that is possible I will try to reproduce it using a simple project. However the error says explicitly that it cannot parse 4.13.1 as a version. Anyway please give me some time.

honnix commented 4 years ago

@marcphilipp You are right about mixing JUnit 5 and JUnit 4. It seems that triggers maven-surefire-plugin to work in a very different way. https://github.com/junit-team/junit5-samples/pull/137/checks?check_run_id=1265044819 reproduced it.

I'm thinking about possible solutions I can take:

honnix commented 4 years ago

Still I think this is somewhat annoying. A patch version bump requires quite some changes in the pom.

marcphilipp commented 4 years ago

Closing since upgrading to 5.7.0 of the JUnit Vintage Engine is a valid fix. Sorry for any inconvenience caused.