Choose a database to match vulnerabilities

There are more tools and options to match the generated SBOM or a package(s) against a database of known vulnerabilities. The main requirements for are use case are:

to be able to match the whole SBOM or a list of packages in batch mode
to be able to check for vulnerabilities during runtime
the solution should work without a need to install a tool or run a server/Docker container
the database should be a well-known data source or fed from most used data sources (like NVD)
the database should contain the information about a vulnerability we need

During the investigation I chose to test these solutions and tools.

Dependency-Check

has a command line tool, maven and gradle plugins
uses the NVD (mirroring the datasources)
can scan a project or a path

Links

Run from command line dependency-check --out . --scan . --format JSON

Run with maven plugin

<plugin>
    <groupId>org.owasp</groupId>
    <artifactId>dependency-check-maven</artifactId>
    <version>8.2.1</version>
    <executions>
        <execution>
            <goals>
                <goal>check</goal>
            </goals>
        </execution>
    </executions>
</plugin>

These generates a report in a format we choose, e.g. JSON, HTML.

It has a dependency-check-maven dependency which can be used to create an own Analyzer and include it into scanning process.

<dependency>
    <groupId>org.owasp</groupId>
    <artifactId>dependency-check-maven</artifactId>
    <version>8.2.1</version>
</dependency>

Also, using the dependency-check-core dependency a scan can be run during runtime, but it mirrors the NVD datasources locally and it needs the path of the packages. I couldn't do a successful scan with this solution.

void checkDependencies() throws ScanAgentException {
    List<Dependency> dependencies = new ArrayList<>();
    Dependency dependency = new Dependency(true);
    dependency.setEcosystem(Ecosystem.JAVA);
    dependency.setPackagePath("...");
    dependency.setName("...");
    dependency.setVersion("...");
    dependency.setActualFilePath("...");
    dependency.setFileName("...");
    dependencies.add(dependency);

    DependencyCheckScanAgent scan = new DependencyCheckScanAgent();
    scan.setDependencies(dependencies);
    scan.setReportFormat(ReportGenerator.Format.HTML);
    scan.setReportOutputDirectory(DependencyCheckService.class.getResource("/resources").getPath());
    scan.execute();
}

Dependency-Track

has an API server (contains an embedded Jetty server)
has a Frontend application (Single Page Application)
it is backed with the most known data sources like NVD, Github Advisories, OSV, VulnDB, etc.
can be run in a Docker container or as executable WAR
has a REST API endpoint to run a scan against a BOM json
it has no public API, we would need to run it to access the REST API

Links

Run in a Docker container

# Downloads the latest Docker Compose file
curl -LO https://dependencytrack.org/docker-compose.yml
docker-compose up -d

I could run the API server in a Docker container and could use the REST API to scan a BOM and get a report.

NVD (National Vulnerability Database)

has a public API to get the vulnerabilities
has a public change history API to mirror the data sources and keep them up-to-date
without requesting an API key the public APIs can only be used with request limits (5 requests in a rolling 30 second window, with API key 50 requests in a rolling 30 second window)

Links

Using the CVE API with a CPE string https://services.nvd.nist.gov/rest/json/cves/2.0?virtualMatchString=cpe:2.3:a:*:json-smart

In NVD datasources some package group names are not matching with the names from the generated SBOM. E.g. in NVD the json-smart package has the json-smart_project group name instead of net.minidev what is in the SBOM. It is not simple to do a query request for some packages, like in the above example where the group name must be omitted because of this cpe:2.3:a:json-smart_project:json-smart -> cpe:2.3:a:*:json-smart. Also, it is not simple to do a query for more packages at a time with the available parameter options, see the API documentation.

OSV (Open Source Vulnerability)

has a public API to get the vulnerabilities with no request limits
the API has endpoints for doing a simple or a batch query to get the package vulnerabilities (1000 packages at a time), it also has an endpoint for getting the details of a vulnerability based on its id
has the osv-scanner command line tool which can scan SBOM files or a path
backed with well-known vulnerability databases, like GitHub Security Advisories Database which also uses more well-known databases like the NVD database

Using the command line tool to scan the SBOM

sv-scanner --sbom bom.json --format json > vulns.json

Using the public API endpoint for a simple query POST https://api.osv.dev/v1/query with group:name name and version parameters

{
  "version": "2.4.8",
  "package": {
    "name": "net.minidev:json-smart",
    "ecosystem": "Maven"
  }
}

or with purl parameter

{
  "package": {
    "ecosystem": "Maven",
    "purl": "pkg:maven/net.minidev/json-smart@2.4.8"
  }
}

Output

{
    "vulns": [
        {
            "id": "GHSA-493p-pfq6-5258",
            "summary": "json-smart Uncontrolled Recursion vulnerabilty",
            "details": "### Impact\nAffected versions of [net.minidev:json-smart](https://github.com/netplex/json-smart-v1) are vulnerable to Denial of Service (DoS) due to a StackOverflowError when parsing a deeply nested JSON array or object.\n\nWhen reaching a ‘[‘ or ‘{‘ character in the JSON input, the code parses an array or an object respectively. It was discovered that the 3PP does not have any limit to the nesting of such arrays or objects. Since the parsing of nested arrays and objects is done recursively, nesting too many of them can cause stack exhaustion (stack overflow) and crash the software.\n\n### Patches\nThis vulnerability was fixed in json-smart version 2.4.9, but the maintainer recommends upgrading to 2.4.10, due to a remaining bug.\n\n### Workarounds\nN/A\n\n### References\n- https://www.cve.org/CVERecord?id=CVE-2023-1370\n- https://nvd.nist.gov/vuln/detail/CVE-2023-1370\n- https://security.snyk.io/vuln/SNYK-JAVA-NETMINIDEV-3369748",
            "aliases": [
                "CVE-2023-1370"
            ],
            "modified": "2023-04-11T01:43:19.744798Z",
            "published": "2023-03-23T20:32:03Z",
            "database_specific": {
                "cwe_ids": [
                    "CWE-674"
                ],
                "github_reviewed": true,
                "severity": "HIGH",
                "github_reviewed_at": "2023-03-23T20:32:03Z",
                "nvd_published_at": null
            },
            "references": [
                {
                    "type": "WEB",
                    "url": "https://github.com/oswaldobapvicjr/jsonmerge/security/advisories/GHSA-493p-pfq6-5258"
                },
                {
                    "type": "ADVISORY",
                    "url": "https://nvd.nist.gov/vuln/detail/CVE-2023-1370"
                },
                {
                    "type": "WEB",
                    "url": "https://github.com/netplex/json-smart-v2/issues/137"
                },
                {
                    "type": "WEB",
                    "url": "https://github.com/netplex/json-smart-v2/commit/5b3205d051952d3100aa0db1535f6ba6226bd87a"
                },
                {
                    "type": "WEB",
                    "url": "https://github.com/netplex/json-smart-v2/commit/e2791ae506a57491bc856b439d706c81e45adcf8"
                },
                {
                    "type": "PACKAGE",
                    "url": "https://github.com/oswaldobapvicjr/jsonmerge"
                },
                {
                    "type": "WEB",
                    "url": "https://research.jfrog.com/vulnerabilities/stack-exhaustion-in-json-smart-leads-to-denial-of-service-when-parsing-malformed-json-xray-427633/"
                },
                {
                    "type": "WEB",
                    "url": "https://security.snyk.io/vuln/SNYK-JAVA-NETMINIDEV-3369748"
                },
                {
                    "type": "WEB",
                    "url": "https://www.cve.org/CVERecord?id=CVE-2023-1370"
                }
            ],
            "affected": [
                {
                    "package": {
                        "name": "net.minidev:json-smart",
                        "ecosystem": "Maven",
                        "purl": "pkg:maven/net.minidev/json-smart"
                    },
                    "ranges": [
                        {
                            "type": "ECOSYSTEM",
                            "events": [
                                {
                                    "introduced": "0"
                                },
                                {
                                    "fixed": "2.4.9"
                                }
                            ]
                        }
                    ],
                    "versions": [
                        "1.0.6.3",
                        "1.0.8",
                        "1.0.9",
                        "1.0.9-1",
                        "1.1",
                        "1.1.1",
                        "1.2",
                        "1.3",
                        "1.3.1",
                        "1.3.2",
                        "1.3.3",
                        "2.0",
                        "2.0-RC1",
                        "2.0-RC2",
                        "2.0-RC3",
                        "2.1.0",
                        "2.1.1",
                        "2.2",
                        "2.2.1",
                        "2.3",
                        "2.3.1",
                        "2.4.1",
                        "2.4.2",
                        "2.4.4",
                        "2.4.5",
                        "2.4.6",
                        "2.4.7",
                        "2.4.8"
                    ],
                    "database_specific": {
                        "source": "https://github.com/github/advisory-database/blob/main/advisories/github-reviewed/2023/03/GHSA-493p-pfq6-5258/GHSA-493p-pfq6-5258.json"
                    }
                }
            ],
            "schema_version": "1.4.0",
            "severity": [
                {
                    "type": "CVSS_V3",
                    "score": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H"
                }
            ]
        }
    ]
}

Using the public API with batch query POST https://api.osv.dev/v1/querybatch

{
  "queries": [
    {
    "version": "2.4.8",
      "package": {
          "name": "net.minidev:json-smart",
          "ecosystem": "Maven"
      }
    }
  ]
}

or with purl paramater

{
  "queries": [
    {
        "package": {
            "ecosystem": "Maven",
            "purl": "pkg:maven/net.minidev/json-smart@2.4.8"
        }
    }
  ]
}

Output

{
    "results": [
        {
            "vulns": [
                {
                    "id": "GHSA-493p-pfq6-5258",
                    "modified": "2023-04-11T01:43:19.744798Z"
                }
            ]
        }
    ]
}

Using the public API to get the details of a vulnerability GET https://api.osv.dev/v1/vulns/GHSA-493p-pfq6-5258 Output

{
    "id": "GHSA-493p-pfq6-5258",
    "summary": "json-smart Uncontrolled Recursion vulnerabilty",
    "details": "### Impact\nAffected versions of [net.minidev:json-smart](https://github.com/netplex/json-smart-v1) are vulnerable to Denial of Service (DoS) due to a StackOverflowError when parsing a deeply nested JSON array or object.\n\nWhen reaching a ‘[‘ or ‘{‘ character in the JSON input, the code parses an array or an object respectively. It was discovered that the 3PP does not have any limit to the nesting of such arrays or objects. Since the parsing of nested arrays and objects is done recursively, nesting too many of them can cause stack exhaustion (stack overflow) and crash the software.\n\n### Patches\nThis vulnerability was fixed in json-smart version 2.4.9, but the maintainer recommends upgrading to 2.4.10, due to a remaining bug.\n\n### Workarounds\nN/A\n\n### References\n- https://www.cve.org/CVERecord?id=CVE-2023-1370\n- https://nvd.nist.gov/vuln/detail/CVE-2023-1370\n- https://security.snyk.io/vuln/SNYK-JAVA-NETMINIDEV-3369748",
    "aliases": [
        "CVE-2023-1370"
    ],
    "modified": "2023-04-11T01:43:19.744798Z",
    "published": "2023-03-23T20:32:03Z",
    "database_specific": {
        "cwe_ids": [
            "CWE-674"
        ],
        "github_reviewed": true,
        "severity": "HIGH",
        "github_reviewed_at": "2023-03-23T20:32:03Z",
        "nvd_published_at": null
    },
    "references": [
        {
            "type": "WEB",
            "url": "https://github.com/oswaldobapvicjr/jsonmerge/security/advisories/GHSA-493p-pfq6-5258"
        },
        {
            "type": "ADVISORY",
            "url": "https://nvd.nist.gov/vuln/detail/CVE-2023-1370"
        },
        {
            "type": "WEB",
            "url": "https://github.com/netplex/json-smart-v2/issues/137"
        },
        {
            "type": "WEB",
            "url": "https://github.com/netplex/json-smart-v2/commit/5b3205d051952d3100aa0db1535f6ba6226bd87a"
        },
        {
            "type": "WEB",
            "url": "https://github.com/netplex/json-smart-v2/commit/e2791ae506a57491bc856b439d706c81e45adcf8"
        },
        {
            "type": "PACKAGE",
            "url": "https://github.com/oswaldobapvicjr/jsonmerge"
        },
        {
            "type": "WEB",
            "url": "https://research.jfrog.com/vulnerabilities/stack-exhaustion-in-json-smart-leads-to-denial-of-service-when-parsing-malformed-json-xray-427633/"
        },
        {
            "type": "WEB",
            "url": "https://security.snyk.io/vuln/SNYK-JAVA-NETMINIDEV-3369748"
        },
        {
            "type": "WEB",
            "url": "https://www.cve.org/CVERecord?id=CVE-2023-1370"
        }
    ],
    "affected": [
        {
            "package": {
                "name": "net.minidev:json-smart",
                "ecosystem": "Maven",
                "purl": "pkg:maven/net.minidev/json-smart"
            },
            "ranges": [
                {
                    "type": "ECOSYSTEM",
                    "events": [
                        {
                            "introduced": "0"
                        },
                        {
                            "fixed": "2.4.9"
                        }
                    ]
                }
            ],
            "versions": [
                "1.0.6.3",
                "1.0.8",
                "1.0.9",
                "1.0.9-1",
                "1.1",
                "1.1.1",
                "1.2",
                "1.3",
                "1.3.1",
                "1.3.2",
                "1.3.3",
                "2.0",
                "2.0-RC1",
                "2.0-RC2",
                "2.0-RC3",
                "2.1.0",
                "2.1.1",
                "2.2",
                "2.2.1",
                "2.3",
                "2.3.1",
                "2.4.1",
                "2.4.2",
                "2.4.4",
                "2.4.5",
                "2.4.6",
                "2.4.7",
                "2.4.8"
            ],
            "database_specific": {
                "source": "https://github.com/github/advisory-database/blob/main/advisories/github-reviewed/2023/03/GHSA-493p-pfq6-5258/GHSA-493p-pfq6-5258.json"
            }
        }
    ],
    "schema_version": "1.4.0",
    "severity": [
        {
            "type": "CVSS_V3",
            "score": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H"
        }
    ]
}

The OSV public API endpoints seems to be the best choice for our requirements. We could use its batch query endpoint to get a list of vulnerability ids for all the packages and after we could call the vulnerability details endpoint for each id and collect the responses.

It can be also useful that from OSV storage we can get all the vulnerabilities for an ecosystem in one zip file. For Maven ecosystem this is the zip https://osv-vulnerabilities.storage.googleapis.com/Maven/all.zip It contains 3477 items currently which are JSON files with the details of each vulnerability. Inside the JSON there is the CVE-... id also under the aliases . As I see, currently all the vulnerabilities in Maven ecosystem are coming from GHSA. Using the GHSA-... id we can get the vulnerability details also from the GitHub advisories https://github.com/advisories/GHSA-493p-pfq6-5258 and there we can find references to other databases, like https://nvd.nist.gov/vuln/detail/CVE-2023-1370 .

Links

vaadin / appsec-kit