jenkinsci / jenkins-test-harness

Unit test framework for Jenkins core and its plugins
https://www.jenkins.io/doc/developer/testing/
MIT License
109 stars 126 forks source link

library leaks junit-api causing test failures in PCT #861

Open jtnord opened 3 days ago

jtnord commented 3 days ago

Jenkins and plugins versions report

Plugin leaks junit-jupiter-api breaking tests in PCT Internal plugin using junit-5 and j-t-h when run in the PCT ```text INFO] Require upper bound dependencies error for org.junit.jupiter:junit-jupiter-api:5.10.3 [test] paths to dependency are: +-com.cloudbees.operations-center.client:operations-center-client:3.27256 +-org.jenkins-ci.main:jenkins-test-harness:2299.v7e8d22e797d3 [test] +-org.junit.jupiter:junit-jupiter-api:5.10.3 [test] (managed) <-- org.junit.jupiter:junit-jupiter-api:5.11.0 [test] and +-com.cloudbees.operations-center.client:operations-center-client:3.27256 +-org.junit.jupiter:junit-jupiter:5.10.3 [test] +-org.junit.jupiter:junit-jupiter-api:5.10.3 [test] (managed) <-- org.junit.jupiter:junit-jupiter-api:5.10.3 [test] and +-com.cloudbees.operations-center.client:operations-center-client:3.27256 +-org.junit.jupiter:junit-jupiter:5.10.3 [test] +-org.junit.jupiter:junit-jupiter-params:5.10.3 [test] (managed) <-- org.junit.jupiter:junit-jupiter-params:5.10.3 [test] +-org.junit.jupiter:junit-jupiter-api:5.10.3 [test] (managed) <-- org.junit.jupiter:junit-jupiter-api:5.10.3 [test] and +-com.cloudbees.operations-center.client:operations-center-client:3.27256 +-org.junit.jupiter:junit-jupiter:5.10.3 [test] +-org.junit.jupiter:junit-jupiter-engine:5.10.3 [test] (managed) <-- org.junit.jupiter:junit-jupiter-engine:5.10.3 [test] +-org.junit.jupiter:junit-jupiter-api:5.10.3 [test] (managed) <-- org.junit.jupiter:junit-jupiter-api:5.10.3 [test] [INFO] for org.junit.jupiter:junit-jupiter-api, upper bounds forces an upgrade from 5.10.3 to 5.11.0 ``` ```text [ERROR] TestEngine with ID 'junit-jupiter' failed to discover tests [ERROR] org.junit.platform.commons.JUnitException: TestEngine with ID 'junit-jupiter' failed to discover tests [ERROR] at org.junit.platform.launcher.core.EngineDiscoveryOrchestrator.discoverEngineRoot(EngineDiscoveryOrchestrator.java:160) ```

What Operating System are you using (both controller, and any agents involved in the problem)?

N/A

Reproduction steps

A plugin that uses junit-5 and JTH whose versions are no aligned (it is not uncommon to update j-t-h without updating the pom as it is not possible in all cases (e.g. retaining compatability, backporting fixes to older jenkins lines etc) mvn test of the plugin will pass, as the enforcer rule does not check test scope dependencies.

However when the plugin is run with the PCT (the war does not need to contain any plugins!) the PCT will attempt to align the test scope dependencies and then the versions will not match.

`mvn -B -V -e -pl :the-plugin -Djenkins.version= -DuseUpperBounds=true hpi:resolve-test-dependencies hpi:test-hpl hpi:test-runtime surefire:test -DoverrideWar=/path/to/the/jenkins.war

Expected Results

no plugins are updated and no error happens

Actual Results

Surefire fails to discover tests

[ERROR] TestEngine with ID 'junit-jupiter' failed to discover tests
[ERROR] org.junit.platform.commons.JUnitException: TestEngine with ID 'junit-jupiter' failed to discover tests
[ERROR]     at org.junit.platform.launcher.core.EngineDiscoveryOrchestrator.discoverEngineRoot(EngineDiscoveryOrchestrator.java:160)

Anything else?

the scope of the jupiter-api should be provided if the library is not present in surefire when running, then you are not using junit-5 tests so the code that uses the api in this plugin should not be loaded. (if the code is loaded for some reason then it should be changed so that the code is only active during junit5 tests)

Are you interested in contributing a fix?

No response

jglick commented 3 days ago

https://github.com/jenkinsci/bom/pull/3720#issuecomment-2404902348 I guess.