Gradle plugin fails to generate reports when configured to use NVD 1.1 datafeed as source rather than NVD API

[jchandler-parsons]

Describe the bug When using the Gradle plugin, configured to pull data from the NVD datafeed or a custom datafeed location, the ODC H2 database is not populated. This causes the "dataExists" check called on the database in the "Engine" to fail and the reports generation to fail.

I used the "--debug" option on Gradle as well as the code here to understand the general behavior. The log output shows that both the META files and the json.gz files are successfully downloaded. I have also verified that the /tmp directory that is created has several megabytes of json.gz files, but the H2 database there and the one copied to ~/.gradle/dependency-check-data/7.0/ are both only ~200 kilobytes. I set the password parameter for the database in Gradle and then used the H2 Shell to open the database, list the tables and count table rows. the "CPEENTRY" table that gets checked by the "dataExists" call using "COUNT_CPE" prepared SQL statement is empty.

The "NoDataException" gets thrown in Engine.ensureDataExists() (Engine.java:1163), and the exception/error message appears in the Gradle log.

I tested this originally in an air-gapped environment with a datafeed mirror, but I have verified the issue still exists when setting the following closure in the Gradle configuration in an Internet connected environment:

  dependencyCheck {
        analyzedTypes = ['jar', 'js']
        formats = ['csv', 'html', 'json']
        nvd {
            datafeedUrl = "https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-{0}.json.gz"
        }
    }

If I switch the Internet connected environment back to using the NVD API endpoint, I successfully get reports.

Version of dependency-check used 9.0.8

Log file Gradle Log (redacted/trimmed)

To Reproduce Steps to reproduce the behavior:

1. apply dependency-check-gradle plugin to a project
2. configure it according to the documentation to use the NVD datafeed instead of the API (see closure above)
3. run gradle dependencyCheckAnalyze --debug on the project
4. get thrown exception/error: org.owasp.dependencycheck.exception.NoDataException: No documents exist

Expected behavior The H2 ODC database cache should have been populated with content. The Gradle plugin wrapped around the code of this project should successfully generate reports.

Additional context I have not gone down further into the code rabbit hole to figure out how the database updates actually occur/are applied, so I'm not sure why the json.gz data isn't read and then written into the database. I know that the latest version of the Gradle plugin is 9.0.9, but I didn't see anything in the release notes related to this issue and version 9.0.8 is the latest currently in the air-gapped environment I need this in.

[jeremylong]

If you were going to use the datafeed - unfortunately the 1.1 data feed from the NVD is not compatible with ODC since the 9.0 version. In fact, the NVD may shut those data feeds down - the reason the project moved to the API is that their data feeds were going to be deprecated and no longer updated at the end of 2023.

To use a data feed you would have to create one using the open-vulnerability-project's vulnz CLI: https://github.com/jeremylong/Open-Vulnerability-Project/tree/main/vulnz#caching-the-nvd-cve-data

[jchandler-parsons]

If I use version 8.x.x (8.4.0 is the next version of the plugin/tool available to me in the air-gapped environment) can I configure it to use the datafeed, and then figure out how to transition to using the API or getting newer datafeed files (vulnz created) later?

[jchandler-parsons]

I found the gradle plugin configuration documentation page in the waybackmachine. My air-gapped box is populating an ODC H2 database now using 8.2.1. I think I have what I need for the time being.