[ERROR] Unable to download meta file: https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-2017.meta

[Cs4r]

Describe the bug Currently Using the Owasp dependency checker for mvn, it is throwing this error below.

Version of dependency-check used 7.4.4

Log file

[ERROR] Failed to execute goal org.owasp:dependency-check-maven:7.4.4:check (default) on project main: Fatal exception(s) analyzing aurum-lottoland-integration-country-features-lambda-main: One or more exceptions occurred during analysis:
[ERROR]     UpdateException: Unable to download meta file: https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-2017.meta
[ERROR]         caused by DownloadFailedException: Download failed, unable to retrieve 'https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-2017.meta'; Error downloading file https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-2017.meta; unable to connect.
[ERROR]         caused by DownloadFailedException: Error downloading file https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-2017.meta; unable to connect.
[ERROR]         caused by DownloadFailedException: Error retrieving https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-2017.meta; received response code 503; Service Unavailable
[ERROR]     NoDataException: No documents exist
[ERROR] -> [Help 1]

[aikebah]

Either there was a temporary outage at NIST, or on the network path towards it.

If you search the issues you'll find plenty of NVD HTTP 503 issues.

The potential solutions:

• use a mirror instead of their live site
• use a central persistent database that you update in a dedicated job

https://jeremylong.github.io/DependencyCheck/data/index.html

[Cs4r]

It started to work again.

The issue was that OWASP server was unavailable (HTTP 503 status code)

[andersonstraube]

Here it only worked again after I inserted the type (--network host) in the Docker command:

FROM: docker run --rm \ -e user=$USER ...

TO: docker run --rm --network host \ -e user=$USER ...

I also started to use the "owasp/dependency-check-action:latest" docker image with the NVD Database Updated.

[aikebah]

@andersonstraube If you encountered the HTTP 503 that would likely be coincidental. The NIST NVD is known to have the occasional issue on reachability resulting in HTTP 503 which will auto-resolve after a while. In my infra we have the nist-data-mirror in use to create our own private mirror. A job is configured on Jenkins to run every two hours to update the mirror using that tool. It runs into error occasionally due to a NIST NVD HTTP 503 and most of the time is back-to-normal on the next build.

[andersonstraube]

@aikebah exactly!! I had some problems with NIST NVD, but in this specific case the content was available with direct access from my system, but inside the DC container it didn't work. It was only resolved when including the "--network host" in command line.

The most curious thing is that I've been running this process for over a year and it was the first time this happened :(

But the idea of having a nist-data-mirror job in your infrastructure is great! Very good!

[mandarpimplapure]

Hello All,

Its still failing for me since last week with below logs :

[ERROR] Unable to continue dependency-check analysis.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  01:14 min
[INFO] Finished at: 2023-07-25T09:04:11Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.owasp:dependency-check-maven:8.3.1:check (default) on project case_management: Fatal exception(s) analyzing case_management: One or more exceptions occurred during analysis:
[ERROR]     UpdateException: Unable to download meta file: https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-modified.meta
[ERROR]         caused by DownloadFailedException: Download failed, unable to retrieve 'https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-modified.meta'; Error downloading file https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-modified.meta; unable to connect.
[ERROR]         caused by DownloadFailedException: Error downloading file https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-modified.meta; unable to connect.
[ERROR]         caused by SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
[ERROR]         caused by ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
[ERROR]         caused by SunCertPathBuilderException: unable to find valid certification path to requested target
[ERROR]     NoDataException: No documents exist

I tried several options like to skip SSL check, mounting the certificate as well as tried to load the certificate in JAVA keystore using keytool but didn't work.

[stuartraetaylor]

Using an API key for NVD API seems to have fixed 504 errors for me:

https://nvd.nist.gov/developers/request-an-api-key

In Gradle:

dependencyCheck {
    // ...
    cve {
        password = "<API KEY>"
    }
}

[aikebah]

Using an API key for NVD API seems to have fixed 504 errors for me:

Seems to... as the API key is meant to be used in different ways.

The API key is meant for the NVD API, which is yet to be supported.

cve.password is for http-basic auth towards the NVD datastreams (e.g. for an in-company hosted http-basic protected mirror).

If you were to hit their rate-limit you would either receive a 403 (forbidden) or a 429 (too many requests) depending on how exactly NVD would've implemented it, I would expect the latter but cannot confirm as I'm not abusively hammering NVD.

You're either just lucky to have requested datastream files only outside of availability issue timewindows of the NVD datastream or the NIST has resolved issues in the availability of their datastream site.

[stuartraetaylor]

Thanks for the explanation. This may be a coincidence in my case. I will continue monitoring the CI jobs to see how it goes over the next week or so.