NullPointerException if top level <testsuite> element does not contain a `name`, and the testcase element does not contain a `classname`

ubergeek42 commented 2 years ago

Version report

Jenkins and plugins versions report:

click to expand jenkins + plugins versions report

``` Jenkins: 2.320 OS: Linux - 4.15.0-147-generic --- ace-editor:1.1 ant:1.12 antisamy-markup-formatter:2.4 apache-httpcomponents-client-4-api:4.5.13-1.0 authentication-tokens:1.4 blueocean:1.25.1 blueocean-autofavorite:1.2.4 blueocean-bitbucket-pipeline:1.25.1 blueocean-commons:1.25.1 blueocean-config:1.25.1 blueocean-core-js:1.25.1 blueocean-dashboard:1.25.1 blueocean-display-url:2.4.1 blueocean-events:1.25.1 blueocean-git-pipeline:1.25.1 blueocean-github-pipeline:1.25.1 blueocean-i18n:1.25.1 blueocean-jira:1.25.1 blueocean-jwt:1.25.1 blueocean-personalization:1.25.1 blueocean-pipeline-api-impl:1.25.1 blueocean-pipeline-editor:1.25.1 blueocean-pipeline-scm-api:1.25.1 blueocean-rest:1.25.1 blueocean-rest-impl:1.25.1 blueocean-web:1.25.1 bootstrap4-api:4.6.0-3 bootstrap5-api:5.1.3-1 bouncycastle-api:2.25 branch-api:2.7.0 build-timeout:1.20 build-user-vars-plugin:1.8 caffeine-api:2.9.2-29.v717aac953ff3 checks-api:1.7.2 cloudbees-bitbucket-branch-source:2.9.11 cloudbees-folder:6.16 command-launcher:1.6 conditional-buildstep:1.4.1 credentials:2.6.2 credentials-binding:1.27 display-url-api:2.3.5 docker-commons:1.17 docker-workflow:1.26 durable-task:1.39 echarts-api:5.2.2-1 email-ext:2.84 external-monitor-job:1.7 favorite:2.3.3 font-awesome-api:5.15.4-1 git:4.10.0 git-client:3.10.0 git-server:1.10 github:1.34.1 github-api:1.133 github-branch-source:2.11.3 google-login:1.6 gradle:1.37.1 handlebars:3.0.8 handy-uri-templates-2-api:2.1.8-1.0 htmlpublisher:1.27 jackson2-api:2.13.0-230.v59243c64b0a5 javadoc:1.6 jaxb:2.3.0.1 jdk-tool:1.5 jenkins-design-language:1.25.1 jira:3.6 jjwt-api:0.11.2-9.c8b45b8bb173 job-dsl:1.78.1 job-restrictions:0.8 jquery-detached:1.2.1 jquery3-api:3.6.0-2 jsch:0.1.55.2 junit:1.53 ldap:2.7 lockable-resources:2.12 mailer:1.34 mapdb-api:1.0.9.0 matrix-auth:2.6.8 matrix-project:1.19 maven-plugin:3.15.1 mercurial:2.15 momentjs:1.1.1 oauth-credentials:0.5 okhttp-api:4.9.2-20211102 pam-auth:1.6.1 parameterized-scheduler:1.0 parameterized-trigger:2.41 permissive-script-security:0.7 pipeline-build-step:2.15 pipeline-github-lib:1.0 pipeline-graph-analysis:1.11 pipeline-input-step:2.12 pipeline-milestone-step:1.3.2 pipeline-model-api:1.9.2 pipeline-model-definition:1.9.2 pipeline-model-extensions:1.9.2 pipeline-rest-api:2.19 pipeline-stage-step:2.5 pipeline-stage-tags-metadata:1.9.2 pipeline-stage-view:2.19 pipeline-utility-steps:2.10.0 plain-credentials:1.7 plugin-util-api:2.5.1 popper-api:1.16.1-2 popper2-api:2.10.2-1 promoted-builds:3.10 pubsub-light:1.16 resource-disposer:0.16 run-condition:1.5 saml:2.0.9 scm-api:2.6.5 script-security:1.78 slack:2.48 snakeyaml-api:1.29.1 sse-gateway:1.24 ssh-credentials:1.19 ssh-slaves:1.33.0 sshd:3.1.0 structs:1.23 subversion:2.15.1 throttle-concurrents:2.4 timestamper:1.13 token-macro:267.vcdaea6462991 trilead-api:1.0.13 variant:1.4 windows-slaves:1.8 workflow-aggregator:2.6 workflow-api:2.47 workflow-basic-steps:2.24 workflow-cps:2633.v6baeedc13805 workflow-cps-global-lib:545.v7b28cce323cf workflow-durable-task-step:1097.veac1aacfbda8 workflow-job:2.42 workflow-multibranch:2.26 workflow-scm-step:2.13 workflow-step-api:2.24 workflow-support:3.8 ws-cleanup:0.39 ```

What Operating System are you using (both controller, and any agents involved in the problem)?

Linux (ubuntu 18.04), docker pipeline job

Reproduction steps

Create a junit.xml file that looks something like (probably malformed/definitely missing attributes, but it's what I'm getting from the software I'm using, and circleci seems happy with it)

<?xml version="1.0" encoding="UTF-8"?>
<testsuite id="815305" tests="9" failures="0">
<testcase id="11829887" name="[debug] test 1" assertions="2" time="181">
</testcase>
<testcase id="11845586" name="[debug] test 2" assertions="1" time="186">
</testcase>
<testcase id="11849855" name="[debug] test 3" assertions="2" time="203">
</testcase>
</testsuite>

Try to add this with a pipeline script step:

junit testResults: 'junit.xml', allowEmptyResults: true

Results

Expected result:

It to work properly/not fail.

Actual result:

[Pipeline] // node
[Pipeline] End of Pipeline
java.lang.NullPointerException
    at hudson.tasks.junit.CaseResult.getPackageName(CaseResult.java:399)
    at hudson.tasks.junit.TestResult.tally(TestResult.java:795)
    at hudson.tasks.junit.JUnitParser$ParseResultCallable.invoke(JUnitParser.java:145)
    at hudson.FilePath.act(FilePath.java:1165)
    at hudson.FilePath.act(FilePath.java:1148)
    at hudson.tasks.junit.JUnitParser.parseResult(JUnitParser.java:107)
    at hudson.tasks.junit.JUnitResultArchiver.parse(JUnitResultArchiver.java:153)
    at hudson.tasks.junit.JUnitResultArchiver.parseAndSummarize(JUnitResultArchiver.java:247)
    at hudson.tasks.junit.pipeline.JUnitResultsStepExecution.run(JUnitResultsStepExecution.java:63)
    at hudson.tasks.junit.pipeline.JUnitResultsStepExecution.run(JUnitResultsStepExecution.java:29)
    at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
Finished: FAILURE

Probable cause/solution:

It looks like there's a null check around the name attribute in SuiteResult.java: https://github.com/jenkinsci/junit-plugin/blob/3cc961f67f1e3d6cccfd2a9974d45ae4feb447cf/src/main/java/hudson/tasks/junit/SuiteResult.java#L215-L224

But a little further down, the name attribute is assigned to the classname of the CaseResult, without checking if it's null first, which is what I think eventually blows up: https://github.com/jenkinsci/junit-plugin/blob/3cc961f67f1e3d6cccfd2a9974d45ae4feb447cf/src/main/java/hudson/tasks/junit/SuiteResult.java#L254-L257

Probably that second instance should use this.name(or the local name directly) instead of suite.attributeValue("name").

timja commented 2 years ago

Interested in providing a PR? With a test ideally

ubergeek42 commented 2 years ago

Interested in providing a PR? With a test ideally

I'm not sure I have any time in the near future to get a java dev env set up and put a fix together. I've worked around it in my use case for the time being by munging my junit xml before passing it off to jenkins.

jenkinsci / junit-plugin