search

jeremylong / DependencyCheck

OWASP dependency-check is a software composition analysis utility that detects publicly disclosed vulnerabilities in application dependencies.
https://owasp.org/www-project-dependency-check/
Apache License 2.0
6.3k stars 1.26k forks source link

Check Yarn vulnerabilities with aggregate goal of maven plugin #6325

Open sbernard31 opened 8 months ago

sbernard31 commented 8 months ago

I tried to integrate dependency-check-maven to my multi-module maven project, all works as expected when using for java.

But in my project I'm using frontend-maven-plugin to build a frontend webapp with yarn.

There is 2 web application in webapp folder of 2 child module (leshan-server-demo / leshan-bsserver-demo) .
Yarn is downloaded by this frontend-maven-plugin.
So I adapted my pom like this : 

        <plugin>
          <groupId>org.owasp</groupId>
          <artifactId>dependency-check-maven</artifactId>
          <configuration>
            <scanSet>
              <fileSet>
                <directory>${project.basedir}/webapp</directory>
                <includes>
                  <include>package.json</include>
                  <include>yarn.lock</include>
                </includes>
              </fileSet>
            </scanSet>
            <pathToYarn>${project.basedir}/webapp/node/yarn/dist/bin/yarn</pathToYarn>
          </configuration>
        </plugin>

See commit : https://github.com/eclipse-leshan/leshan/commit/c611bcfa852e330af13a3cc27bbb14b83fde253e

This works when I use mvn dependency-check:check (I mean I see js vulnerabilities).
But this doesn't work when using mvn dependency-check:aggregate. (in that case I just see java vulnerabilities)

(I know that documentation says to not use ${project.basedir} but I was not able to make it works without this.

I succeed to workaround this by adding this configuration on root pom.xml : 

<profile>
  <id>aggregate</id>
  <build>
    <plugins>
      <plugin>
        <groupId>org.owasp</groupId>
        <artifactId>dependency-check-maven</artifactId>
        <configuration>
          <scanSet>
            <fileSet>
              <directory>${project.basedir}/leshan-server-demo/webapp</directory>
              <includes>
                <include>package.json</include>
                <include>yarn.lock</include>
              </includes>
            </fileSet>
            <fileSet>
              <directory>${project.basedir}/leshan-bsserver-demo/webapp</directory>
              <includes>
                <include>package.json</include>
                <include>yarn.lock</include>
              </includes>
            </fileSet>
          </scanSet>
          <pathToYarn>${project.basedir}/leshan-server-demo/webapp/node/yarn/dist/bin/yarn</pathToYarn>
        </configuration>
      </plugin>
    </plugins>
  </build>
</profile>

(commit https://github.com/eclipse-leshan/leshan/commit/4e247194c90ce97de8ce314ca36a49b48323cc73) And now it works when using : mvn -Paggregate dependency-check:aggregate But the solution is clearly not elegant and so I guess I missed something :thinking:

Please let me know, if I did something wrong ? :pray:

If you want to reproduce : 

# Get owasp branch in official leshan repository
git clone -b owasp git@github.com:eclipse-leshan/leshan.git leshan-owasp
cd leshan-owasp
# build the project without tests. (this will download node/yarn)
mvn clean install -DskipTests
# See result with and without workaround (aggregate profile)
mvn dependency-check:aggregate 
mvn -Paggregate dependency-check:aggregate
sbernard31 commented 8 months ago

I tried to make it works on my jenkins instances (using jenkins pipeline).

And there is more problems than what I thought.

I faced this error (only on jenkins, locally it works): 

[WARNING] The Yarn Audit Analyzer has been disabled. Yarn executable was not found.

(My 2 cts : I feel this warn :point_up: should be more verbose (path of yarn, error returns, ...)) I double check the yarn path and I almost sure it is OK.
After investigation I understood that dependency-check-maven, try a yarn --help to check if yarn is found. So I tried to launch this command in my jenkins pipeline and get 

$ ./leshan-server-demo/webapp/node/yarn/dist/bin/yarn --help
Yarn requires Node.js 4.0 or higher to be installed.

In my case Node is also installed by frontend-maven-plugin. In my case at : ${project.basedir}/leshan-server-demo/webapp/node/node. But there is no way to set the node path with something like pathToYarn.

(Why it works locally ? because I have node is available in my local machine PATH)

I tried to workaround it in by : 

export PATH=./leshan-server-demo/webapp/node:$PATH && ./leshan-server-demo/webapp/node/yarn/dist/bin/yarn --help

which works ! So I tried : 

sh '''export PATH=./leshan-server-demo/webapp/node:$PATH && mvn -B -Paggregate  dependency-check:aggregate'''

But I get : 

[WARNING] An unexpected error occurred during analysis of '/home/jenkins/agent/workspace/leshan-test/leshan-server-demo/webapp/yarn.lock' (Yarn Audit Analyzer): No value present
[WARNING] An unexpected error occurred during analysis of '/home/jenkins/agent/workspace/leshan-test/leshan-bsserver-demo/webapp/yarn.lock' (Yarn Audit Analyzer): No value present
[ERROR] 
java.util.NoSuchElementException: No value present
    at java.util.Optional.get (Optional.java:148)
    at org.owasp.dependencycheck.analyzer.YarnAuditAnalyzer.fetchYarnAuditJson (YarnAuditAnalyzer.java:241)
    at org.owasp.dependencycheck.analyzer.YarnAuditAnalyzer.analyzePackage (YarnAuditAnalyzer.java:281)
    at org.owasp.dependencycheck.analyzer.YarnAuditAnalyzer.analyzeDependency (YarnAuditAnalyzer.java:106)
    at org.owasp.dependencycheck.analyzer.AbstractAnalyzer.analyze (AbstractAnalyzer.java:131)
    at org.owasp.dependencycheck.AnalysisTask.call (AnalysisTask.java:88)
    at org.owasp.dependencycheck.AnalysisTask.call (AnalysisTask.java:37)
    at java.util.concurrent.FutureTask.run (FutureTask.java:264)
    at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1128)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:628)
    at java.lang.Thread.run (Thread.java:829)
[ERROR] 
java.util.NoSuchElementException: No value present
    at java.util.Optional.get (Optional.java:148)
    at org.owasp.dependencycheck.analyzer.YarnAuditAnalyzer.fetchYarnAuditJson (YarnAuditAnalyzer.java:241)
    at org.owasp.dependencycheck.analyzer.YarnAuditAnalyzer.analyzePackage (YarnAuditAnalyzer.java:281)
    at org.owasp.dependencycheck.analyzer.YarnAuditAnalyzer.analyzeDependency (YarnAuditAnalyzer.java:106)
    at org.owasp.dependencycheck.analyzer.AbstractAnalyzer.analyze (AbstractAnalyzer.java:131)
    at org.owasp.dependencycheck.AnalysisTask.call (AnalysisTask.java:88)
    at org.owasp.dependencycheck.AnalysisTask.call (AnalysisTask.java:37)
    at java.util.concurrent.FutureTask.run (FutureTask.java:264)
    at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1128)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:628)
    at java.lang.Thread.run (Thread.java:829)

I will probably investigate other vulnerabilities check solution (https://github.com/eclipse-leshan/leshan/issues/1566) But in all case if you want to make works this use cases, I can try to help.

jeremylong commented 8 months ago

CycloneDX plugins maybe viable. You can then setup dependency-track to monitor the SBOMs.

jeremylong commented 8 months ago

Another option might be to configure the front-end plugin's installation directory:

https://github.com/eirslett/frontend-maven-plugin?tab=readme-ov-file#installation-directory

Then also configure these to be on the path so that node and yarn.

sbernard31 commented 8 months ago

Another option might be to configure the front-end plugin's installation directory: https://github.com/eirslett/frontend-maven-plugin?tab=readme-ov-file#installation-directory Then also configure these to be on the path so that node and yarn.

I'm not sure how it could help as I can not set nodePath in dependency-check-maven ? I maybe missed something ?

CycloneDX plugins maybe viable. You can then setup dependency-track to monitor the SBOMs.

I understand that maybe a more future proof approach is to use some tools to generate standard SBOMs, then using other tools to check vulnerabilities from that SBOMs.

When you say dependency-track, are you talking about :

jeremylong commented 8 months ago

dependency-track platform

sbernard31 commented 8 months ago

I understand that you implicitly says that this project (DependencyCheck) is more or less the past and the future is more about using SBOM standard like (cycloneDX) and related tooling ?

dependency-track platform

The "problem" with dependency-track is that I need to deploy it. Maybe It's not a good idea but I was searching more a build tool which give me a list of issue/warning (ideally integrated with maven)

jeremylong commented 8 months ago

This is related to https://github.com/jeremylong/DependencyCheck/issues/5432

jeremylong commented 8 months ago

does the frontend maven plugin specify the path to node via an ENV variable or config option for yarn?

sbernard31 commented 8 months ago

This is related to https://github.com/jeremylong/DependencyCheck/issues/5432

It looks like a little bit.

does the frontend maven plugin specify the path to node via an ENV variable or config option for yarn?

I'm not sure. This is frontend-maven-plugin which install node and yarn, you can choose where to install using https://github.com/eirslett/frontend-maven-plugin#installing-node-and-yarn.

I think yarn is executed via : https://github.com/eirslett/frontend-maven-plugin/blob/5ad126f96120c42e4f5d7856af57434485c26a66/frontend-plugin-core/src/main/java/com/github/eirslett/maven/plugins/frontend/lib/YarnExecutor.java#L13-L21

The yarn config is created here : https://github.com/eirslett/frontend-maven-plugin/blob/5ad126f96120c42e4f5d7856af57434485c26a66/frontend-plugin-core/src/main/java/com/github/eirslett/maven/plugins/frontend/lib/FrontendPluginFactory.java#L67-L69

And Paths come from : https://github.com/eirslett/frontend-maven-plugin/blob/5ad126f96120c42e4f5d7856af57434485c26a66/frontend-maven-plugin/src/main/java/com/github/eirslett/maven/plugins/frontend/mojo/AbstractFrontendMojo.java#L94-L95

Mainly : 

/**
 * The base directory for installing node and npm.
 */
@Parameter(property = "installDirectory", required = false)
protected File installDirectory;
sbernard31 commented 8 months ago

(On my side I will experiment others tools like trivy)

jeremylong commented 8 months ago

One thing you may find with tools like trivy is that some JAR files may not be correctly identified. Problematic JARs would be shaded/uber/one JAR files and possibly those built using something other than Maven (which could be the transitive dependencies used in your app). I'm not saying things won't be identified correctly - I don't use trivy specifically. I have found these to be an issue with other tools so it is something to watch out for. One of the main reasons is that Maven does a great thing by embedding the pom.xml into the JAR which really helps with identification. Other build management tools do not do this.

sbernard31 commented 8 months ago

Thx for warning me :pray:.

As I said ideally, I would like to just use a maven plugin to check dependencies vulnerabilities.

But It seems that industry is moving to something like tooling for generating SBOM, then use that SBOM for cheking vulnerability (and also some other task like checking licenses)

So I was thinking that I will easily find little specialized tools to generate SBOM and other tools specialized to do vulnerabilities check and some other to do tools check licenses.

In my case I find plugin to generate SBOM :

And didn't find a small tool to check vulnerabilities from that SBOM. (Maybe I can use Trivy : https://aquasecurity.github.io/trivy/v0.48/docs/target/sbom/)

But it seems there is rather big all-in-one tooling :

jeremylong commented 8 months ago

If you ran your build with -X does it output the full yarn command used during the build? I haven't had time to dig into the maven-frontend-plugin - I may have to soon for an issue outside of this plugin. There has to be a way to get ODC to work with this.

Can you have the front-end plugin run yarn audit?

sbernard31 commented 8 months ago

If you ran your build with -X does it output the full yarn command used during the build?

Yes, I can see it : [DEBUG] Executing command line [/home/sbernard/git/leshan/leshan-bsserver-demo/webapp/node/yarn/dist/bin/yarn]

Unfold for full logs 
[INFO] --- frontend-maven-plugin:1.13.4:yarn (yarn install) @ leshan-bsserver-demo ---
[DEBUG] Configuring mojo com.github.eirslett:frontend-maven-plugin:1.13.4:yarn from plugin realm ClassRealm[plugin>com.github.eirslett:frontend-maven-plugin:1.13.4, parent: jdk.internal.loader.ClassLoaders$AppClassLoader@55054057]
[DEBUG] Configuring mojo 'com.github.eirslett:frontend-maven-plugin:1.13.4:yarn' with basic configurator -->
[DEBUG]   (f) project = MavenProject: org.eclipse.leshan:leshan-bsserver-demo:2.0.0-SNAPSHOT @ /home/sbernard/git/leshan/leshan-bsserver-demo/pom.xml
[DEBUG]   (f) repositorySystemSession = org.eclipse.aether.DefaultRepositorySystemSession@50910df1
[DEBUG]   (f) session = org.apache.maven.execution.MavenSession@42373389
[DEBUG]   (f) skipTests = true
[DEBUG]   (f) testFailureIgnore = false
[DEBUG]   (f) workingDirectory = /home/sbernard/git/leshan/leshan-bsserver-demo/webapp
[DEBUG]   (f) yarnInheritsProxyConfigFromMaven = true
[DEBUG]   (f) execution = com.github.eirslett:frontend-maven-plugin:1.13.4:yarn {execution: yarn install}
[DEBUG] -- end configuration --
[INFO] Running 'yarn ' in /home/sbernard/git/leshan/leshan-bsserver-demo/webapp
[DEBUG] Executing command line [/home/sbernard/git/leshan/leshan-bsserver-demo/webapp/node/yarn/dist/bin/yarn]
[INFO] yarn install v1.22.19
[INFO] [1/4] Resolving packages...
[INFO] success Already up-to-date.
[INFO] Done in 0.23s.
[DEBUG] Exit value 0
[INFO] 
[INFO] --- frontend-maven-plugin:1.13.4:yarn (yarn build) @ leshan-bsserver-demo ---
[DEBUG] Configuring mojo com.github.eirslett:frontend-maven-plugin:1.13.4:yarn from plugin realm ClassRealm[plugin>com.github.eirslett:frontend-maven-plugin:1.13.4, parent: jdk.internal.loader.ClassLoaders$AppClassLoader@55054057]
[DEBUG] Configuring mojo 'com.github.eirslett:frontend-maven-plugin:1.13.4:yarn' with basic configurator -->
[DEBUG]   (f) arguments = build
[DEBUG]   (f) environmentVariables = {MAVEN_OUTPUT_DIR=/home/sbernard/git/leshan/leshan-bsserver-demo/target/classes/webapp, VITE_APP_COMMIT_ID=3c92a00a8aa6b0f83a0bc96d013bcd1a6723f455, VITE_APP_VERSION=2.0.0-SNAPSHOT}
[DEBUG]   (f) project = MavenProject: org.eclipse.leshan:leshan-bsserver-demo:2.0.0-SNAPSHOT @ /home/sbernard/git/leshan/leshan-bsserver-demo/pom.xml
[DEBUG]   (f) repositorySystemSession = org.eclipse.aether.DefaultRepositorySystemSession@50910df1
[DEBUG]   (f) session = org.apache.maven.execution.MavenSession@42373389
[DEBUG]   (f) skipTests = true
[DEBUG]   (f) testFailureIgnore = false
[DEBUG]   (f) workingDirectory = /home/sbernard/git/leshan/leshan-bsserver-demo/webapp
[DEBUG]   (f) yarnInheritsProxyConfigFromMaven = true
[DEBUG]   (f) execution = com.github.eirslett:frontend-maven-plugin:1.13.4:yarn {execution: yarn build}
[DEBUG] -- end configuration --
[INFO] Running 'yarn build' in /home/sbernard/git/leshan/leshan-bsserver-demo/webapp
[DEBUG] Executing command line [/home/sbernard/git/leshan/leshan-bsserver-demo/webapp/node/yarn/dist/bin/yarn, build]

Can you have the front-end plugin run yarn audit?

Yep I did it : https://github.com/eclipse-leshan/leshan/blob/5fb771466ae2e0a9a8c85aab7762009a4709920a/build-config/demo-build-config/pom.xml#L104-L121

Maven output looks like : 

[INFO] --- frontend-maven-plugin:1.13.4:yarn (yarn audit) @ leshan-server-demo ---
[INFO] Running 'yarn audit' in /home/sbernard/git/leshan/leshan-server-demo/webapp
[INFO] yarn audit v1.22.19
[INFO] ┌───────────────┬──────────────────────────────────────────────────────────────┐
[INFO] │ moderate      │ Vite XSS vulnerability in `server.transformIndexHtml` via    │
[INFO] │               │ URL payload                                                  │
[INFO] ├───────────────┼──────────────────────────────────────────────────────────────┤
[INFO] │ Package       │ vite                                                         │
[INFO] ├───────────────┼──────────────────────────────────────────────────────────────┤
[INFO] │ Patched in    │ >=5.0.5                                                      │
[INFO] ├───────────────┼──────────────────────────────────────────────────────────────┤
[INFO] │ Dependency of │ vite                                                         │
[INFO] ├───────────────┼──────────────────────────────────────────────────────────────┤
[INFO] │ Path          │ vite                                                         │
[INFO] ├───────────────┼──────────────────────────────────────────────────────────────┤
[INFO] │ More info     │ https://www.npmjs.com/advisories/1095178                     │
[INFO] └───────────────┴──────────────────────────────────────────────────────────────┘
[INFO] 1 vulnerabilities found - Packages audited: 253
[INFO] Severity: 1 Moderate
[INFO] Done in 1.08s.
[INFO] ------------------------------------------------------------------------
jeremylong commented 8 months ago

I had an idea about how I might be able to make ODC work with this. I'm going to have to do some testing on https://github.com/eclipse-leshan/leshan/commit/c611bcfa852e330af13a3cc27bbb14b83fde253e

sbernard31 commented 8 months ago

On my case, I finally find an alternative :

chadlwilson commented 1 month ago

My guess is that the root of the problem here is likely mainly that ODC doesn't work with Yarn 2+ (Berry), not something specific to Maven. (although the error reporting needs some work to make it easier to find the problem as you tend to get java.util.NoSuchElementException: No value present for a number of different errors)

4215

4894

sbernard31 commented 1 month ago

My guess is that the root of the problem here is likely mainly that ODC doesn't work with Yarn 2+ (Berry), not something specific to Maven.

Just to mention all my tests/use cases above :point_up: use Yarn Classic (v1).