search

jeremylong / DependencyCheck

OWASP dependency-check is a software composition analysis utility that detects publicly disclosed vulnerabilities in application dependencies.
https://owasp.org/www-project-dependency-check/
Apache License 2.0
6.48k stars 1.29k forks source link

https://services.nvd.nist.gov/ endopint is giving 503 Service Unavailable #6758

Open hpriya19 opened 5 months ago

hpriya19 commented 5 months ago

OWASP Client version: 9.0.8

We are seeing the services endpoint timeout from last couple of days

RUN /tmp/dependency-check/bin/dependency-check.sh --updateonly  --nvdApiKey ${NVD_API_KEY} --nvdApiDelay 3000
Picked up JAVA_TOOL_OPTIONS: -Dhttp.proxyHost=*** -Dhttp.proxyPort=80 -Dhttps.proxyHost=*** -Dhttps.proxyPort=80
[INFO] Checking for updates
[WARN] NVD API request failures are occurring; retrying request for the 5 time
[WARN] NVD API request failures are occurring; retrying request for the 6 time
[WARN] NVD API request failures are occurring; retrying request for the 7 time
[WARN] NVD API request failures are occurring; retrying request for the 5 time
[WARN] NVD API request failures are occurring; retrying request for the 6 time
[WARN] NVD API request failures are occurring; retrying request for the 7 time

We have verified the nvdApiKey is valid. The curl response to the endpoint gives the response below

curl -H "Accept: application/json" -H "apiKey:******" -v https://services.nvd.nist.gov/rest/json/cves/2.0\?cpeName\=cpe:2.3:o:microsoft:windows_10:1607:\*:\*:\*:\*:\*:\*:\*
* Uses proxy env variable https_proxy ***
*   Trying 10.68.69.53...
* TCP_NODELAY set
* Connected to <proxy> (10.68.69.53) port 80 (#0)
* allocate connect buffer!
* Establish HTTP proxy tunnel to services.nvd.nist.gov:443
> CONNECT services.nvd.nist.gov:443 HTTP/1.1
> Host: services.nvd.nist.gov:443
> User-Agent: curl/7.61.1
> Proxy-Connection: Keep-Alive
> 
< HTTP/1.0 200 Connection established
< 
* Proxy replied 200 to CONNECT request
* CONNECT phase completed!
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* CONNECT phase completed!
* CONNECT phase completed!
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, [no content] (0):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, [no content] (0):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, [no content] (0):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, [no content] (0):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, [no content] (0):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use http/1.1
* Server certificate:
*  subject: CN=*.nvd.nist.gov
*  start date: May 31 01:13:03 2024 GMT
*  expire date: Aug 29 01:13:02 2024 GMT
*  subjectAltName: host "services.nvd.nist.gov" matched cert's "*.nvd.nist.gov"
*  issuer: C=US; O=Let's Encrypt; CN=R3
*  SSL certificate verify ok.
* TLSv1.3 (OUT), TLS app data, [no content] (0):
> GET /rest/json/cves/2.0?cpeName=cpe:2.3:o:microsoft:windows_10:1607:*:*:*:*:*:*:* HTTP/1.1
> Host: services.nvd.nist.gov
> User-Agent: curl/7.61.1
> Accept: application/json
> apiKey:****
> 
* TLSv1.3 (IN), TLS handshake, [no content] (0):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, [no content] (0):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS app data, [no content] (0):
< HTTP/1.1 503 Service Unavailable
< content-length: 107
< cache-control: no-cache
< content-type: text/html
< 
<html><body><h1>503 Service Unavailable</h1>
No server is available to handle this request.
</body></html>
* Connection #0 to host *** left intact

Is there a change in Service endpoint? Can we get some assistance with this.

jeremylong commented 4 months ago

@chadlwilson I'll also be improving the error reporting around the NVD API Key (see ovc/55678e58084496f39ca6a92a9fc7dc90b5e82231). I've seen at least one CI with an incorrectly set NVD API Key provide an empty string - which can really cause some confusion and problems.

chadlwilson commented 4 months ago

Yeah, I've seen issues before with Gradle system properties and project properties being read incorrectly (or at the wrong part of Gradle lifecycle) so it'll definitely help to have a bit more "it looks like you're doing something wrong..." here.

sinattieng commented 4 days ago

Good morning all, I have a big performance issue related to this topic the API 2.0 https://services.nvd.nist.gov/rest/json/cves/2.0 during the last 3/4 days has been returning HTTP code 503 most interrogation from plugin and the process of dependency-check needs up to 1h for finishing....this is unacceptable. I'm using the latest version of the plugin 11.1.0 with NVD API key. I have no others particular configuration on it, some one has some suggestion?

Thank you in advance.

in-fke commented 4 days ago

Good morning all, I have a big performance issue related to this topic the API 2.0

All I can say: same here ...

dspaeth-breuni commented 3 days ago

Is it possible to use https://api.vulncheck.com/v3/backup/nist-nvd2 (more info https://docs.vulncheck.com/community/nist-nvd/nvd-2) ? Instead of https://services.nvd.nist.gov/rest/json/cves/2.0 in order to circumvent this issue?

ftiercelin commented 3 days ago

@dspaeth-breuni I suppose so, there's a configuration item called nvdApiEndpoint, see https://jeremylong.github.io/DependencyCheck/dependency-check-maven/check-mojo.html

dspaeth-breuni commented 3 days ago

@ftiercelin thanks for your reply. The vulncheck API needs "Authorization: Bearer " the nist api uses the header "apiKey: ", so can I change this too?

in-fke commented 3 days ago

The vulncheck API needs "Authorization: Bearer "

If this is not feasible via the client, you would have to set up some outbound reverse proxy (i.e. with nginx) that enriches your request accordingly.

jeremylong commented 3 days ago

We accept PRs...

geeky-programer commented 2 days ago

Facing the same issues, 503 errors on all requests

ftiercelin commented 2 days ago

We accept PRs...

This is not as simple as I thought. It's not just about shifting from header 'apiKey:{key}' to header 'Authorization: Bearer {key}' The Vulncheck API isn't working exactly as the NVD one.

I think we need a new type of client in package io.github.jeremylong.openvulnerability.client to handle this new API. Then we could work on using this new client type in this project.

I have created an enhancement request: https://github.com/jeremylong/Open-Vulnerability-Project/issues/231

cc @dspaeth-breuni