search

junit-team / junit4

A programmer-oriented testing framework for Java.
https://junit.org/junit4
Eclipse Public License 1.0
8.53k stars 3.29k forks source link

Class naming appears to prevent full parallelization #1709

Closed 7ep closed 3 years ago

7ep commented 3 years ago

In short: the names of classes impact whether they will all be run in parallel.

Longer description: This is a web application written in Kotlin, using Gradle and Junit4. I have configured the system to run tests in parallel. Everything was running fine, but then I noticed a problem after a recent commit. I have an 8-core CPU - I should have four tests running in parallel, and that was happening, but then suddenly it wasn't after one commit. I examined the commit for hours - there was no obvious problem.

I tried messing with the tests, and hit paydirt - if I created a new class with a similar name, it would again run in parallel. Apparently, Junit4 cares about class naming.

I tried investigating the Junit 4 codebase but didn't find an obvious culprit, I'm too unfamilar.

I've distilled the essence of the problem, to ease debugging. If you examine the commit https://github.com/7ep/r3z/commit/5bd49d44e7e8977217726a769ab3d740eae84b5f you will see my workaround fix to the problem - I renamed the classes and they are again running fully in parallel. But if you review the preceeding commit, before the rename, only two run in parallel.

To see the problem - clone the software, checkout that commit, and run gradlew uitests

It will run successfully, fully parallel.

If you checkout the preceeding commit and run that command, only two will run concurrently, followed by the third.

Dependencies: Java on the path, Chrome, Firefox

marcphilipp commented 3 years ago

Since you're using Gradle's maxParallelForks this issue is unrelated to JUnit 4. I recommend you post a question on StackOverflow to get help figuring out the root cause.

7ep commented 3 years ago

Alright, thanks @marcphilipp