search

jenkinsci / junit-plugin

Allows JUnit-format test results to be published
https://plugins.jenkins.io/junit
MIT License
76 stars 335 forks source link

AggregatedTestResultPublisherTest fails with Jenkins 2.434 #588

Closed MarkEWaite closed 10 months ago

MarkEWaite commented 10 months ago

Jenkins and plugins versions report

Environment ```text Jenkins: 2.434 OS: Linux - 6.2.0-36-generic Java: 21.0.1 - Eclipse Adoptium (OpenJDK 64-Bit Server VM) --- PrioritySorter:5.0.0 adoptopenjdk:1.5 analysis-model-api:11.13.0 ant:497.v94e7d9fffa_b_9 antisamy-markup-formatter:162.v0e6ec0fcfcf6 apache-httpcomponents-client-4-api:4.5.14-208.v438351942757 authentication-tokens:1.53.v1c90fd9191a_b_ badge:1.9.1 basic-branch-build-strategies:81.v05e333931c7d bootstrap5-api:5.3.2-2 bouncycastle-api:2.29 branch-api:2.1135.v8de8e7899051 build-blocker-plugin:1.7.9 build-name-setter:2.4.0 build-timeout:1.31 buildresult-trigger:0.18 caffeine-api:3.1.8-133.v17b_1ff2e0599 checks-api:2.0.2 cloudbees-bitbucket-branch-source:856.v04c46c86f911 cloudbees-folder:6.858.v898218f3609d cloudbees-jenkins-advisor:358.v58972d19b_1f0 clover:4.14.1.577.vb_ff5a_f69b_37b_ code-coverage-api:4.99.0 command-launcher:107.v773860566e2e commons-lang3-api:3.13.0-62.v7d18e55f51e2 commons-text-api:1.11.0-94.v3e1f4a_926e49 conditional-buildstep:1.4.3 config-file-provider:959.vcff671a_4518b_ configuration-as-code:1746.vf1673cfe690a copyartifact:722.v0662a_9b_e22a_c coverage:1.5.0 credentials:1309.v8835d63eb_d8a_ credentials-binding:642.v737c34dea_6c2 dark-theme:372.v79b_02c754b_29 data-tables-api:1.13.8-1 description-setter:1.10 design-library:264.vea_e31791b_3c9 display-url-api:2.200.vb_9327d658781 docker-commons:439.va_3cb_0a_6a_fb_29 docker-workflow:572.v950f58993843 durable-task:523.va_a_22cf15d5e0 echarts-api:5.4.3-1 elastic-axis:464.va_7ed499b_9d75 embeddable-build-status:412.v09da_db_1dee68 envinject:2.908.v66a_774b_31d93 envinject-api:1.199.v3ce31253ed13 external-monitor-job:215.v2e88e894db_f8 font-awesome-api:6.4.2-1 forensics-api:2.3.0 git:5.2.1 git-client:4.5.0 git-forensics:2.0.0 git-parameter:0.9.19 git-server:99.va_0826a_b_cdfa_d gitea:1.4.5 github:1.37.3.1 github-api:1.318-461.v7a_c09c9fa_d63 github-branch-source:1752.vc201a_0235d80 github-scm-filter-aged-refs:31.ve3b_ca_fc71d5b_ github-scm-trait-commit-skip:0.4.0 gitlab-api:5.3.0-91.v1f9a_fda_d654f gitlab-branch-source:684.vea_fa_7c1e2fe3 gitlab-plugin:1.7.16 groovy-postbuild:2.5 h2-api:11.1.4.199-12.v9f4244395f7a_ handy-uri-templates-2-api:2.1.8-22.v77d5b_75e6953 http_request:1.18 ignore-committer-strategy:1.0.4 implied-labels:281.v1322d123c2b_4 instance-identity:185.v303dc7c645f9 ionicons-api:56.v1b_1c8c49374e jackson2-api:2.15.3-372.v309620682326 jacoco:3.3.5 jakarta-activation-api:2.0.1-3 jakarta-mail-api:2.0.1-3 javadoc:243.vb_b_503b_b_45537 javax-activation-api:1.2.0-6 javax-mail-api:1.6.2-9 jaxb:2.3.9-1 jdk-tool:73.vddf737284550 jersey2-api:2.41-133.va_03323b_a_1396 jira:3.11 jjwt-api:0.11.5-77.v646c772fddb_0 jquery3-api:3.7.1-1 jsch:0.2.8-65.v052c39de79b_2 junit:1240.vf9529b_881428 junit-realtime-test-reporter:135.vf92a_7fe68b_15 label-verifier:1.4 mailer:463.vedf8358e006b_ mapdb-api:1.0.9-28.vf251ce40855d markdown-formatter:95.v17a_965e696ee matrix-auth:3.2.1 matrix-project:818.v7eb_e657db_924 maven-plugin:3.23 metrics:4.2.18-442.v02e107157925 mina-sshd-api-common:2.11.0-86.v836f585d47fa_ mina-sshd-api-core:2.11.0-86.v836f585d47fa_ monitoring:1.95.0-rc524.88e73a_e5cdfd naginator:1.421.v85c11e2db_507 nodelabelparameter:1.12.0 okhttp-api:4.11.0-157.v6852a_a_fa_ec11 parallel-test-executor:418.v24f9a_141d726 parameterized-trigger:787.v665fcf2a_830b_ pipeline-build-step:516.v8ee60a_81c5b_9 pipeline-github:2.8-155.8eab375ac9f8 pipeline-github-lib:42.v0739460cda_c4 pipeline-graph-analysis:202.va_d268e64deb_3 pipeline-graph-view:205.vb_8e3a_b_51f12e pipeline-groovy-lib:689.veec561a_dee13 pipeline-input-step:477.v339683a_8d55e pipeline-maven:1362.vee39a_d4b_02b_1 pipeline-maven-api:1362.vee39a_d4b_02b_1 pipeline-milestone-step:111.v449306f708b_7 pipeline-model-api:2.2151.ve32c9d209a_3f pipeline-model-definition:2.2151.ve32c9d209a_3f pipeline-model-extensions:2.2151.ve32c9d209a_3f pipeline-stage-step:305.ve96d0205c1c6 pipeline-stage-tags-metadata:2.2151.ve32c9d209a_3f pipeline-utility-steps:2.16.0 plain-credentials:143.v1b_df8b_d3b_e48 platformlabeler:2325.v8f074df8840d plugin-util-api:3.6.0 pollscm:1.5 preSCMbuildstep:71.v1f2990a_37e27 prism-api:1.29.0-9 promoted-builds:936.va_571a_a_b_f8da_5 resource-disposer:0.23 run-condition:1.7 scm-api:683.vb_16722fb_b_80b_ script-security:1294.v99333c047434 snakeyaml-api:2.2-111.vc6598e30cc65 ssh-agent:346.vda_a_c4f2c8e50 ssh-credentials:308.ve4497b_ccd8f4 ssh-slaves:2.916.vd17b_43357ce4 sshd:3.312.v1c601b_c83b_0e structs:325.vcb_307d2a_2782 support-core:1366.v9d076592655d swarm:3.41 testng-plugin:835.v51ed3da_fcc35 text-finder:1.26 theme-manager:215.vc1ff18d67920 throttle-concurrents:2.14 timestamper:1.26 token-macro:384.vf35b_f26814ec trilead-api:2.84.v72119de229b_7 variant:60.v7290fc0eb_b_cd versioncolumn:224.v3b_5dc11d87f1 view-job-filters:369.ve0513a_a_f5524 warnings-ng:10.5.1 workflow-aggregator:596.v8c21c963d92d workflow-api:1283.v99c10937efcb_ workflow-basic-steps:1042.ve7b_140c4a_e0c workflow-cps:3806.va_3a_6988277b_2 workflow-durable-task-step:1289.v4d3e7b_01546b_ workflow-job:1360.vc6700e3136f5 workflow-multibranch:756.v891d88f2cd46 workflow-scm-step:415.v434365564324 workflow-step-api:639.v6eca_cd8c04a_a_ workflow-support:865.v43e78cc44e0d ws-cleanup:0.45 xshell:203.v80a_c62d7886f ```

What Operating System are you using (both controller, and any agents involved in the problem)?

Ubuntu Linux 22.04 on the controller with Java 11, 17, or 21

Reproduction steps

  1. Run the tests from the master branch with Jenkins 2.434 `mvn clean -Djenkins.version=2.434 -Dtest=AggregatedTestResultPublisherTest verify
  2. Confirm that two tests fail with: 
    
[ERROR] Tests run: 3, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 12.61 s <<< FAILURE! -- in hudson.tasks.test.AggregatedTestResultPublisherTest
[ERROR] hudson.tasks.test.AggregatedTestResultPublisherTest.testResultsAndAggregatedTestResults -- Time elapsed: 7.497 s <<< ERROR!
org.htmlunit.ElementNotFoundException: elementName=[a] attributeName=[<text>] attributeValue=[junit #1]
    at org.htmlunit.html.HtmlPage.getAnchorByText(HtmlPage.java:694)
    at hudson.tasks.test.helper.TestResultsPage.hasLinkToTestResultOfBuild(TestResultsPage.java:17)
    at hudson.tasks.test.AggregatedTestResultPublisherTest.testResultsAndAggregatedTestResults(AggregatedTestResultPublisherTest.java:129)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at org.jvnet.hudson.test.JenkinsRule$1.evaluate(JenkinsRule.java:655)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.lang.Thread.run(Thread.java:829)

[ERROR] hudson.tasks.test.AggregatedTestResultPublisherTest.aggregatedTestResultsOnly -- Time elapsed: 2.279 s <<< ERROR! org.htmlunit.ElementNotFoundException: elementName=[a] attributeName=[] attributeValue=[junit #1] at org.htmlunit.html.HtmlPage.getAnchorByText(HtmlPage.java:694) at hudson.tasks.test.helper.TestResultsPage.hasLinkToTestResultOfBuild(TestResultsPage.java:17) at hudson.tasks.test.AggregatedTestResultPublisherTest.aggregatedTestResultsOnly(AggregatedTestResultPublisherTest.java:66) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.jvnet.hudson.test.JenkinsRule$1.evaluate(JenkinsRule.java:655) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.lang.Thread.run(Thread.java:829)

[INFO] [INFO] Results: [INFO] [ERROR] Errors: [ERROR] AggregatedTestResultPublisherTest.aggregatedTestResultsOnly:66 » ElementNotFound elementName=[a] attributeName=[] attributeValue=[junit #1] [ERROR] AggregatedTestResultPublisherTest.testResultsAndAggregatedTestResults:129 » ElementNotFound elementName=[a] attributeName=[] attributeValue=[junit #1] [INFO] [ERROR] Tests run: 3, Failures: 0, Errors: 2, Skipped: 0



The same tests pass with `jenkins.version=2.433`

### Expected Results

Tests should pass

### Actual Results

Tests fail

### Anything else?

Have not yet bisected to the Jenkins core commit that introduced the issue.

### Are you interested in contributing a fix?

_No response_
MarkEWaite commented 10 months ago

Detected in the plugin bill of materials pull request:

MarkEWaite commented 10 months ago

The plugin tests will need additional refinements to adapt to the changes in the Jenkins core pull request: