Java container: not all vulerabilites found by grype are detected

turing85 commented 1 year ago

Current Behavior

I built a container from this github repositry, generated a cyclonedx-sbom with syft (0.91.0) and analyzed the SBOM with grype (0.69.0) as well as dependency-track (4.8.2). Grype finds 83 vulnerabilites, dependency-track finds 3.

Steps to Reproduce

clone https://github.com/ConSol/super-simple-storage-solution

git clone https://github.com/ConSol/super-simple-storage-solution
cd super-simple-storage-solution

build the application

./mvwn -pl services/rest-api --also-make clean package

build a container containing the artifact:

buildah build --file dockerfiles/Dockerfile.quarkus-distroless --tag foo services/rest-api

generate a CycloneDX SBOM with syft:

syft packages -o cyclonedx-json --scope all-layers foo:latest > /tmp/cyclone-syft.json

analyze the SBOM with grype:
```
grype sbom:/tmp/cyclone-syft.json
```

Observe the vulnerabilities grype finds:

✔ Scanned for vulnerabilities     [83 vulnerability matches]  
 ├── by severity: 17 critical, 25 high, 12 medium, 11 low, 18 negligible
 └── by status:   3 fixed, 80 not-fixed, 0 ignored

Start the bundled dependency-track container:

podman run -p 9080:8080 --name dependencytrack -m 8192m -v dependency-track:/data --rm --detach docker.io/dependencytrack/bundled:4.8.2

For first startup: Wait until the container fetched all vulnerabilites
Create a project, upload /tmp/cyclone-syft.json
Wait for analysis to complete
Observe that dependency-track only finds three vulnerabilities:
Observe that the netty-vulnerability, as well as the okhttp vulnerability are not found by grype (not sure if this a followup-error, or if this is an issue in grype)

Expected Behavior

dependency-track should find the same vulnerabilities as grype does.

Additional information

I suspect that the problem is related to the CPEs generated by syft. If we search the CVEs found by grype in the vulneraiblity-database of dependency-track, then the affected CPEs do not match up with the CPEs of the application. Enabling fuzzy CPE matching does find more vulnerabilites, but they are different from the ones reported by grype.

Dependency-Track Version

4.8.2

Dependency-Track Distribution

Container Image

Database Server

H2

Database Server Version

No response

Browser

Google Chrome

Checklist

[X] I have read and understand the contributing guidelines
[X] I have checked the existing issues for whether this defect was already reported

nscuro commented 1 year ago

Thanks for providing steps to reproduce, much appreciated!

For reference, this is the output I'm getting from Grype:

Grype Output

$ grype sbom:cyclone-syft.json
 ✔ Vulnerability DB                [no update available]
 ✔ Scanned for vulnerabilities     [84 vulnerability matches]
   ├── by severity: 17 critical, 24 high, 14 medium, 11 low, 18 negligible
   └── by status:   3 fixed, 81 not-fixed, 0 ignored
NAME                                                      INSTALLED              FIXED-IN     TYPE          VULNERABILITY        SEVERITY
arc                                                       3.2.6.Final                         java-archive  CVE-2005-2992        Low
arc                                                       3.2.6.Final                         java-archive  CVE-2005-2945        Low
com.fasterxml.jackson.core.jackson-databind               2.15.2                              java-archive  CVE-2023-35116       Medium
com.google.guava.failureaccess                            1.0.1                               java-archive  CVE-2023-2976        High
com.google.guava.failureaccess                            1.0.1                               java-archive  CVE-2020-8908        Low
com.google.guava.guava                                    32.0.0-jre                          java-archive  CVE-2023-2976        High
com.google.guava.guava                                    32.0.0-jre                          java-archive  CVE-2020-8908        Low
guava                                                     32.0.0-jre                          java-archive  CVE-2023-2976        High
guava                                                     32.0.0-jre             32.0.0       java-archive  GHSA-7g45-4rm6-3mm3  Medium
guava                                                     32.0.0-jre             32.0.0       java-archive  GHSA-5mg8-w23w-74h3  Low
guava                                                     32.0.0-jre                          java-archive  CVE-2020-8908        Low
io.smallrye.reactive.smallrye-mutiny-vertx-amqp-client    3.5.0                               java-archive  CVE-2022-37832       Critical
io.smallrye.reactive.smallrye-mutiny-vertx-amqp-client    3.5.0                               java-archive  CVE-2018-15529       High
io.smallrye.reactive.smallrye-mutiny-vertx-amqp-client    3.5.0                               java-archive  CVE-2013-0136        High
io.smallrye.reactive.smallrye-mutiny-vertx-auth-common    3.5.0                               java-archive  CVE-2022-37832       Critical
io.smallrye.reactive.smallrye-mutiny-vertx-auth-common    3.5.0                               java-archive  CVE-2018-15529       High
io.smallrye.reactive.smallrye-mutiny-vertx-auth-common    3.5.0                               java-archive  CVE-2013-0136        High
io.smallrye.reactive.smallrye-mutiny-vertx-bridge-common  3.5.0                               java-archive  CVE-2022-37832       Critical
io.smallrye.reactive.smallrye-mutiny-vertx-bridge-common  3.5.0                               java-archive  CVE-2018-15529       High
io.smallrye.reactive.smallrye-mutiny-vertx-bridge-common  3.5.0                               java-archive  CVE-2013-0136        High
io.smallrye.reactive.smallrye-mutiny-vertx-core           3.5.0                               java-archive  CVE-2022-37832       Critical
io.smallrye.reactive.smallrye-mutiny-vertx-core           3.5.0                               java-archive  CVE-2018-15529       High
io.smallrye.reactive.smallrye-mutiny-vertx-core           3.5.0                               java-archive  CVE-2013-0136        High
io.smallrye.reactive.smallrye-mutiny-vertx-runtime        3.5.0                               java-archive  CVE-2022-37832       Critical
io.smallrye.reactive.smallrye-mutiny-vertx-runtime        3.5.0                               java-archive  CVE-2018-15529       High
io.smallrye.reactive.smallrye-mutiny-vertx-runtime        3.5.0                               java-archive  CVE-2013-0136        High
io.smallrye.reactive.smallrye-mutiny-vertx-uri-template   3.5.0                               java-archive  CVE-2022-37832       Critical
io.smallrye.reactive.smallrye-mutiny-vertx-uri-template   3.5.0                               java-archive  CVE-2018-15529       High
io.smallrye.reactive.smallrye-mutiny-vertx-uri-template   3.5.0                               java-archive  CVE-2013-0136        High
io.smallrye.reactive.smallrye-mutiny-vertx-web            3.5.0                               java-archive  CVE-2022-37832       Critical
io.smallrye.reactive.smallrye-mutiny-vertx-web            3.5.0                               java-archive  CVE-2018-15529       High
io.smallrye.reactive.smallrye-mutiny-vertx-web            3.5.0                               java-archive  CVE-2013-0136        High
io.smallrye.reactive.smallrye-mutiny-vertx-web-common     3.5.0                               java-archive  CVE-2022-37832       Critical
io.smallrye.reactive.smallrye-mutiny-vertx-web-common     3.5.0                               java-archive  CVE-2018-15529       High
io.smallrye.reactive.smallrye-mutiny-vertx-web-common     3.5.0                               java-archive  CVE-2013-0136        High
jackson-databind                                          2.15.2                              java-archive  CVE-2023-35116       Medium
libc6                                                     2.31-13+deb11u6        (won't fix)  deb           CVE-2023-4813        Medium
libc6                                                     2.31-13+deb11u6        (won't fix)  deb           CVE-2023-4806        Medium
libc6                                                     2.31-13+deb11u6                     deb           CVE-2019-9192        Negligible
libc6                                                     2.31-13+deb11u6                     deb           CVE-2019-1010025     Negligible
libc6                                                     2.31-13+deb11u6                     deb           CVE-2019-1010024     Negligible
libc6                                                     2.31-13+deb11u6                     deb           CVE-2019-1010023     Negligible
libc6                                                     2.31-13+deb11u6                     deb           CVE-2019-1010022     Negligible
libc6                                                     2.31-13+deb11u6                     deb           CVE-2018-20796       Negligible
libc6                                                     2.31-13+deb11u6                     deb           CVE-2010-4756        Negligible
libexpat1                                                 2.2.10-2+deb11u5                    deb           CVE-2013-0340        Negligible
libfreetype6                                              2.10.4+dfsg-1+deb11u1               deb           CVE-2022-31782       Negligible
libgcc-s1                                                 10.2.1-6               (won't fix)  deb           CVE-2023-4039        Medium
libglib2.0-0                                              2.66.8-1               (won't fix)  deb           CVE-2023-29499       High
libglib2.0-0                                              2.66.8-1               (won't fix)  deb           CVE-2023-32665       Medium
libglib2.0-0                                              2.66.8-1               (won't fix)  deb           CVE-2023-32611       Medium
libglib2.0-0                                              2.66.8-1                            deb           CVE-2012-0039        Negligible
libharfbuzz0b                                             2.7.4-1                (won't fix)  deb           CVE-2023-25193       High
libharfbuzz0b                                             2.7.4-1                (won't fix)  deb           CVE-2022-33068       Medium
libjpeg62-turbo                                           1:2.0.6-4              (won't fix)  deb           CVE-2021-46822       Medium
libpcre3                                                  2:8.39-13                           deb           CVE-2019-20838       Negligible
libpcre3                                                  2:8.39-13                           deb           CVE-2017-7246        Negligible
libpcre3                                                  2:8.39-13                           deb           CVE-2017-7245        Negligible
libpcre3                                                  2:8.39-13                           deb           CVE-2017-16231       Negligible
libpcre3                                                  2:8.39-13                           deb           CVE-2017-11164       Negligible
libpng16-16                                               1.6.37-3                            deb           CVE-2021-4214        Negligible
libpng16-16                                               1.6.37-3                            deb           CVE-2019-6129        Negligible
libstdc++6                                                10.2.1-6               (won't fix)  deb           CVE-2023-4039        Medium
libuuid1                                                  2.36.1-8+deb11u1                    deb           CVE-2022-0563        Negligible
mutiny                                                    2.3.1                               java-archive  CVE-2022-37832       Critical
mutiny                                                    2.3.1                               java-archive  CVE-2018-15529       High
mutiny                                                    2.3.1                               java-archive  CVE-2013-0136        High
okhttp                                                    3.14.9                              java-archive  CVE-2023-0833        Medium
okio                                                      1.17.2                              java-archive  CVE-2023-3635        High
okio                                                      1.17.2                 3.4.0        java-archive  GHSA-w33c-445m-f8w7  Medium
openjdk-17-jre-headless                                   17.0.7+7-1~deb11u1                  deb           CVE-2023-22041       Medium
openjdk-17-jre-headless                                   17.0.7+7-1~deb11u1                  deb           CVE-2023-22049       Low
openjdk-17-jre-headless                                   17.0.7+7-1~deb11u1                  deb           CVE-2023-22045       Low
openjdk-17-jre-headless                                   17.0.7+7-1~deb11u1                  deb           CVE-2023-22044       Low
openjdk-17-jre-headless                                   17.0.7+7-1~deb11u1                  deb           CVE-2023-22036       Low
openjdk-17-jre-headless                                   17.0.7+7-1~deb11u1                  deb           CVE-2023-22006       Low
org.apache.maven.resolver.maven-resolver-api              1.9.13                              java-archive  CVE-2021-26291       Critical
org.apache.maven.resolver.maven-resolver-connector-basic  1.9.13                              java-archive  CVE-2021-26291       Critical
org.apache.maven.resolver.maven-resolver-impl             1.9.13                              java-archive  CVE-2021-26291       Critical
org.apache.maven.resolver.maven-resolver-named-locks      1.9.13                              java-archive  CVE-2021-26291       Critical
org.apache.maven.resolver.maven-resolver-spi              1.9.13                              java-archive  CVE-2021-26291       Critical
org.apache.maven.resolver.maven-resolver-transport-http   1.9.10                              java-archive  CVE-2021-26291       Critical
org.apache.maven.resolver.maven-resolver-transport-wagon  1.9.13                              java-archive  CVE-2021-26291       Critical
org.apache.maven.resolver.maven-resolver-util             1.9.13                              java-archive  CVE-2021-26291       Critical

Few observations:

There are loads of false positives in Grype's output:
- CVE-2013-0136, CVE-2018-15529, CVE-2022-37832 appear 9 times each for mutiny components (the reactive framework that powers Quarkus). The CVEs however are raised against Mutiny "Monitoring Appliance", not the mutiny framework. That's 9 * 3 = 27 FPs.
- CVE-2021-26291 appears 8 times for org.apache.maven.resolver.* components. While the CVE was raised against Maven (< 3.8.1), you'll find that Maven resolver is a different (sub-)project of Maven, with its own versioning (latest is 1.9.16), and not at all affected by this. 8 FPs.
- CVE-2005-2992, CVE-2005-2945 appear for the arc component, which is the CDI / IoC implementation that Quarkus uses. That framework did not even exist in 2005, the CVEs were raised against different component named arc. 2 FPs.
Syft tends to fuzz CPEs, so it will include multiple CPEs for the same component via syft:cpe23 properties. I'd classify this as a symptom of how unreliable CPE matching is (see https://owasp.org/blog/2022/09/13/sbom-forum-recommends-improvements-to-nvd). From an identity perspective, a single component having multiple CPEs does not make sense, hence why CycloneDX does not support this natively. DT does not evaluate Syft's CPE properties, and thus may miss findings that Grype is able to find based on those.
DT per default uses Sonatype OSS Index to identity vulnerabilities in components with valid PURL. While no service is perfect, OSS Index has historically done a stellar job of preventing false positives.
- OSS Index does however not support Debian right now (https://ossindex.sonatype.org/ecosystems)

I'll not do an investigation for the entire result list, but I hope you get the idea. Comparing raw numbers of findings between scanners is not a reliable way to evaluate which one did a better job.

In order for us to improve DT, we'll need more specific information. What are valid vulnerabilities that DT was not able to find, but Grype was? With that info, we can investigate. But a list with that many false positives is not helpful.

turing85 commented 1 year ago

Hi! Thanks for the response and explanation.

To see which of the vulnerabilities may be FP or not discovered, I generated a sbom through the cyclonedx-maven-plugin:

cd services/rest-api
mkdir --parent target/sbom
mvn \                                                                                                               
  --define projectType=application \
  --define schemaVersion=1.4 \
  --define includeBomSerialNumber=true \
  --define includeCompileScope=true \
  --define includeProvidedScope=true \
  --define includeRuntimeScope=true \
  --define includeSystemScope=true \
  --define includeTestScope=false \
  --define includeLicenseText=false \
  --define outputFormat=json \
  --define outputName=target/sbom/sbom \
  org.cyclonedx:cyclonedx-maven-plugin:2.7.9:makeBom

I hope that the output of this plugin is somewhat trustworthy. Uploading and analyzing this SBOM to dependency-track results in the following vulnerabilities:

So the results seem identical save for the graalvm-sdk vulerability. I think the issue can be closed :slightly_smiling_face:

github-actions[bot] commented 1 year ago

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

DependencyTrack / dependency-track