Using surefire or failsafe causes multiple deployments

[hneubauer]

Using maven-surefire or maven-failsafe (2.22+) means that this extension no longer works correctly. Instead of a single deployment, another one is made for each test. As long as you don't have to use these two plugins, all is ok. It seems the problem exists since version 2.22 of the plugins (introducing junit 5)

[MartinHlavna]

I am also experiencing this bug. Any workarounds?

[hneubauer]

I'm using failsafe 2.19.1 now for integration tests. The version of failsafe must be not higher, because since 2.22 failsafe uses junit5 and running arquillian tests with failsafe and the vintange engine (because Arquillian doesn't support junit5) results in multiple deployments. With 2.19.1 you have only one deployment.

For junit 5 tests I'm using surefire 2.22.2.

Both together with different maven goals work as expected.

[vjmp06]

I'm use arquillian-extension-suite to execute all test in once deployment

and the problem multilpe deployements with junit 4.7 and maven-surefire-plugin 2.4.3, do you has any solution this? Without more Thanks!

[hneubauer]

I haven't tried that. But I assume that there are always multiple deployments when the Arquillian extension suite runs with JUnit5 in the vintage engine. So it's important to do the Arquillian tests with surefire or failsafe in a version that does not yet work with JUnit5.

If you still want to have additional JUnit5 tests, you have the option to do the integration tests and the JUnit tests with different plugins. That's what I did in the end. The integration tests with an old failsafe version, the JUnit5 tests with a newer surefire version.

The JUnit5 tests then run on the executing machine, of course, while the integration tests run on the machine with the single Arquillian deployment.

[lprimak]

I am seeing the same (incorrect) behavior and multiple deployments. Using JUnit 5 and latest surefire plugin, using Arquillian 1.7.0-Alpha5 which supports JUnit 5. Any workarounds? This extension is useless without it.

[ingwarsw]

@lprimak Could you prepare minimal test project so I could try reproducing issue?

Im not using Arquillian for long time so its not tested with new versions..

[lprimak]

@ingwarsw I don't think there is such a thing as "minimal" with Arquillian but I will try as self-contained as possible :)

[lprimak]

@ingwarsw I am working on a very minimal reproducer, pretty much done with it, I think it's very simple and demonstrates the point, however, it requires Payara and I am going to have to do test containers and docker to make this as simple as possible. Stay tuned, should have something in a few hours

[lprimak]

@ingwarsw I have created a self-contained test application.

• Clone https://github.com/flowlogix/test-arq-suite
• run it (docker on localhost: mvn test
• run it (docker on non-localhost): mvn -Dpayara.adminHost=$(docker-machine ip) test

The test fails because it catches two deployments, where there should be only one. If/when the suite runs correctly, all tests should pass.

[lprimak]

@ingwarsw have you had a chance to look at above reproducer? Does it work for you? Do you need help with it? I am anxious to see whether I can get the suite extension to work properly :) thank you!

[ingwarsw]

@lprimak After few hours of debugging jupiter seems its not that easy..

Each class follows full install/uninstall extension rule..

So if you will have 100k tests in one class suite extension will work.. But for each class there will be new deploy (thats why your test app shows 2 deploys instead of 4)..

Im still trying to find any way to hack into jupiter system to know that all classes finished.. If I will be able to do that maybe I will be able to fix that line https://github.com/arquillian/arquillian-core/blob/master/junit5/core/src/main/java/org/jboss/arquillian/junit5/ArquillianTestClassLifecycleManager.java#L42 (and corresponding code in beforeTestClassPhase..

[lprimak]

@ingwarsw thanks for looking at it, and glad the reproducer works for you. Just one note, as there would not be 4 deployments in this test. Arquillian, by default, is set up with one deployment per class, not per test. If you remove the @ArquillianSuiteDeployment you still see 2 deployments.

Also, if there needs to be a PR to Arquillian, they are pretty responsive as well.

[ingwarsw]

@lprimak Im almost there..

I just need PR for arquillian.. (And Im wondering how many things I will broke)

[lprimak]

@ingwarsw thank you!

[lprimak]

Can you post the PR when you submit it so I can try it please?

[lprimak]

@ingwarsw Can you back-port this to Java 8? It fails the Arquillian CI testing

[lprimak]

Hmm, it fails with this in JDK 8 but not JDK 11:

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.22.2:test (default-test) on project arquillian-test-spi: Unable to generate classpath: org.apache.maven.artifact.resolver.MultipleArtifactsNotFoundException: Missing:
[ERROR] ----------
[ERROR] 1) org.apache.maven.surefire:surefire-junit4:jar:2.22.2
[ERROR] 
[ERROR]   Try downloading the file manually from the project website.
[ERROR] 
[ERROR]   Then, install it using the command: 
[ERROR]       mvn install:install-file -DgroupId=org.apache.maven.surefire -DartifactId=surefire-junit4 -Dversion=2.22.2 -Dpackaging=jar -Dfile=/path/to/file
[ERROR] 
[ERROR]   Alternatively, if you host your own repository you can deploy the file there: 
[ERROR]       mvn deploy:deploy-file -DgroupId=org.apache.maven.surefire -DartifactId=surefire-junit4 -Dversion=2.22.2 -Dpackaging=jar -Dfile=/path/to/file -Durl=[url] -DrepositoryId=[id]
[ERROR] 
[ERROR]   Path to dependency: 
[ERROR]     1) dummy:dummy:jar:1.0
[ERROR]     2) org.apache.maven.surefire:surefire-junit4:jar:2.22.2
[ERROR] 
[ERROR] ----------
[ERROR] 1 required artifact is missing.
[ERROR] 
[ERROR] for artifact: 
[ERROR]   dummy:dummy:jar:1.0
[ERROR] 
[ERROR] from the specified remote repositories:
[ERROR]   jboss-public-repository (https://repository.jboss.org/nexus/content/groups/public/, releases=true, snapshots=false),
[ERROR]   central (https://repo.maven.apache.org/maven2, releases=true, snapshots=false)
[ERROR] 
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <args> -rf :arquillian-test-spi

[ingwarsw]

But i didnt change anything there.. seems like network issue..

[lprimak]

Well, never mind, I see you fixed it!!!

[lprimak]

Tested for both JUnit 5 and 4, and works correctly!

[lprimak]

Please close this issue

[ingwarsw]

It took a bit but now it should work with junit5 (with new versions of arquillian only)

[MartinHlavna]

Hello,

I have migrated project to use latest arquillian and junit 5 yet the problem still persists...

@lprimak can you please share the versions of dependencies you are using?

[lprimak]

• Latest Arauillian (1.7.0-Alpha9)
• Latest surefire / failsafe (3.0.0-M5)
• Latest JUnit 5

[MartinHlavna]

FYI I have found my problem. For future reference:

Our project structure consists of one "core" project of our product platform. This core projects also contains some Test classes where the tests are prepared (no @Test annotations in these classes). This is the project where @Deployment method is. All test classes inherits from this class.

The implementation project also enables any applicable tests by overriding test methods and annotates them with @Test annotations. Looks like extension can't find Deployment method from core project as it is specified only as library. Solution is to configure deploymentClass using arquillian.xml.

[lprimak]

Have you tried https://maven.apache.org/surefire/maven-surefire-plugin/test-mojo.html#dependenciesToScan @MartinHlavna ?

[MartinHlavna]

@lprimak no... that would probably work too.