search

jeremylong / DependencyCheck

OWASP dependency-check is a software composition analysis utility that detects publicly disclosed vulnerabilities in application dependencies.
https://owasp.org/www-project-dependency-check/
Apache License 2.0
6.29k stars 1.26k forks source link

Invalid/inapropriate JUnit report file when using in combination with Jenkins multi-module build and xUnit plugin #4469

Closed haumacher closed 2 years ago

haumacher commented 2 years ago

I'm trying to use the dependency-check plugin in a multi-module maven build from Jenkins CI. I try to produce a test report in Jenkins from all generated JUnit report files in all directories */target/dependency-check-junit.xml using the xUnit plugin. I'm facing two problems:

  1. The xUnit plugin complains that generated the JUnit report is invalid, because "Attribute 'errors' must appear on element 'testsuite'.". This results in no report generated at all.
  2. When fixing problem 1 with an XSLT afterburner, I get a check report, but from the report, I cannot determine the concrete module that has the problematic dependency. This is because the JUnit report does not contain the module that causes the problem. In a build with 100+ modules this is really cumbersome to find the source of the problematic dependency:

grafik

Version of dependency-check used

   <groupId>org.owasp</groupId>
   <artifactId>dependency-check-maven</artifactId>
   <version>7.1.0</version>

Log file

Not the log file of the dependency check, but the Jenkins log of the xUnit plugin that tries to consume its result:

INFO: Processing JUnit
INFO: [JUnit] - 2 test report file(s) were found with the pattern '*/target/dependency-check-junit.xml' relative to '/var/lib/jenkins/workspace/CheckDependencies' for the testing framework 'JUnit'.
WARNING: The file '/var/lib/jenkins/workspace/CheckDependencies/ext.com.meterware.httpunit/target/dependency-check-junit.xml' is an invalid file.
WARNING: At line 1 of file:/var/lib/jenkins/workspace/CheckDependencies/ext.com.meterware.httpunit/target/dependency-check-junit.xml:cvc-complex-type.4: Attribute 'errors' must appear on element 'testsuite'.
WARNING: At line 1 of file:/var/lib/jenkins/workspace/CheckDependencies/ext.com.meterware.httpunit/target/dependency-check-junit.xml:cvc-complex-type.4: Attribute 'errors' must appear on element 'testsuite'.
WARNING: At line 1 of file:/var/lib/jenkins/workspace/CheckDependencies/ext.com.meterware.httpunit/target/dependency-check-junit.xml:cvc-complex-type.4: Attribute 'errors' must appear on element 'testsuite'.
WARNING: At line 1 of file:/var/lib/jenkins/workspace/CheckDependencies/ext.com.meterware.httpunit/target/dependency-check-junit.xml:cvc-complex-type.4: Attribute 'errors' must appear on element 'testsuite'.
ERROR: Step ‘Publish xUnit test result report’ failed: The result file '/var/lib/jenkins/workspace/CheckDependencies/ext.com.meterware.httpunit/target/dependency-check-junit.xml' for the metric 'JUnit' is not valid. The result file has been skipped.
aikebah commented 2 years ago

@haumacher I've created a different proposal on how to integrate the module(s) in which an issue was found that better fits the project. The modules are added to the information in the system-err field.

Samples of the result:

For a (not advised) setup where dependency-check:check is run for each module like you appear to do from your screenshot (results in duplicate 'failing tests' for the same dependency across multiple modules (one for each module where the dependency resides)

DependencyCheck-Check-xUnit-Jenkins-TestReport

For a (recommended) setup where mvn dependency-check:aggregate runs only for the top-level parent of the multimodule project (results in a single 'failing test' referencing all the modules in which the dependency resides)

DependencyCheck-Aggregate-xUnit-Jenkins-TestReport

Would like to hear your opinion on this alternate approach. You can build a snapshot version from the issue-4469 branch