Getting 403 or 404 exception with NVD API key

[1-mrudulaahire]

Hello, All!! After jeremylong suggetion moved to 10.0.1. But using this with NVD API key getting 403/404 exception. Though using api key came through email only. How to get know that my API activated or not?

[jeremylong]

See https://github.com/jeremylong/Open-Vulnerability-Project/tree/main/vulnz#api-key-is-used-and-a-403-or-404-error-occurs

[1-mrudulaahire]

@jeremylong Didn't get JSON with curl commad, tried commad from CMD. but already requested NVD API key three times. Getting exception every time.

[lgolubenkobit]

See https://github.com/jeremylong/Open-Vulnerability-Project/tree/main/vulnz#api-key-is-used-and-a-403-or-404-error-occurs

We are having this error when update using apiKey. I've tried doing curl from cmd and works ok. On pom we have

                <nvdApiDelay>16000</nvdApiDelay>
                <nvdValidForHours>8</nvdValidForHours>
                <connectionTimeout>120000</connectionTimeout>

[adam-siklosi]

Same here, just checked that I have a valid api key. It started happening this night. Yesterday it was all good, didn't change anything in my setup. Version 10.0.1 cli / gradle plugin, default settings apart from the credentials.

[1-mrudulaahire]

See https://github.com/jeremylong/Open-Vulnerability-Project/tree/main/vulnz#api-key-is-used-and-a-403-or-404-error-occurs

We are having this error when update using apiKey. I've tried doing curl from cmd and works ok. On pom we have

                <nvdApiDelay>16000</nvdApiDelay>
                <nvdValidForHours>8</nvdValidForHours>
                <connectionTimeout>120000</connectionTimeout>

Hi, @lgolubenkobit. Did we have any log or action anything. That's shows our NVD API key is valid? It would be really helpful

[mdzt-axi]

I started getting this error last night too.

[MidasBOC]

We've experienced the same problem. We found this information in the log:

2024-07-09 08:15:32,287 io.github.jeremylong.openvulnerability.client.nvd.NvdCveClient:359 DEBUG - Response: \\\

403 Forbidden\

Request forbidden by administrative rules. \\

[adam-siklosi]

I think the issue is this: https://github.com/jeremylong/DependencyCheck?tab=readme-ov-file#mandatory-upgrade-notice Using 10.0.2 locally seems to solved my problem.

Edit: just updated to 10.0.2 in our repositories and now all work properly.

[aggelos-gamvrinos-sympower]

Having the same issue with the gradle plugin. Checked the Nvd api key and it is valid. Upgrading the plugin to 10.0.2 didn't solve it

[MidasBOC]

I think the issue is this: https://github.com/jeremylong/DependencyCheck?tab=readme-ov-file#mandatory-upgrade-notice Using 10.0.2 locally seems to solved my problem.

Edit: just updated to 10.0.2 in our repositories and now all work properly.

Bumping the version worked for us. Thank you very much.

[jeremylong]

@aggelos-gamvrinos-sympower did you validate that the API key is still valid? See the documentation on how to validate the key is correct.

[somera]

Works fine with 10.0.1 and 10.0.2 for mee.

[aggelos-gamvrinos-sympower]

Edit: Sorry ended up working with 10.0.2. We're using libs and we hadn't properly bumped the version

[david-pulkowski]

Just making a note.

Our API key is valid. We were also getting the 403 error in our Jenkins CI/CD nightly job after our jenkins plugin brought in the latest OWASP version, 10.0.2.

Troubleshooting: if I download the zip from this project and run it manually on the server, (running the same command as we have in our job, where it fails) the command is successful and updates the DB properly.

So for us, I think its something with how the jenkins plugin caches old versions of the plugin?

I will have to let the job rerun in a few hours, or let the nightly job run again.

[jeremylong]

As I just found out - it might be how the apiKey is being provided. I'll release an update soon that will improve the error reporting around the API key.

[nico-mcalley]

I updated org.owasp:dependency-check-gradle from 10.0.0 -> 10.0.2 and it worked again.

[chadlwilson]

Yes, as noted as mandatory at https://github.com/jeremylong/DependencyCheck/issues/6817 :-)

[phs-glenfordham]

hi @jeremylong eagerly awaiting that API key update as we're still getting that generic error after upgrading to 10.0.2.

also, is anyone else getting a 400 error when using the CURL command provided here? even if I remove the API key entirely I get a 400 error, seems like the request is malformed? edit: looks like Windows CURL through the command prompt doesn't like the command, but it works fine in Bash, interesting!

C:\Users\glenfordham>curl -H "Accept: application/json" -H "apiKey: redacted" -v https://services.nvd.nist.gov/rest/json/cves/2.0\?cpeName\=cpe:2.3:o:microsoft:windows_10:1607:\*:\*:\*:\*:\*:\*:\*
* Host services.nvd.nist.gov:443 was resolved.
* IPv6: (none)
* IPv4: 54.85.30.225
*   Trying 54.85.30.225:443...
* Connected to services.nvd.nist.gov (54.85.30.225) port 443
* schannel: disabled automatic use of client certificate
* ALPN: curl offers http/1.1
* ALPN: server accepted http/1.1
* using HTTP/1.x
> GET /rest/json/cves/2.0\?cpeName\=cpe:2.3:o:microsoft:windows_10:1607:\*:\*:\*:\*:\*:\*:\* HTTP/1.1
> Host: services.nvd.nist.gov
> User-Agent: curl/8.7.1
> Accept: application/json
> apiKey: redacted
>
* Request completely sent off
* schannel: remote party requests renegotiation
* schannel: renegotiating SSL/TLS connection
* schannel: SSL/TLS connection renegotiated
* schannel: remote party requests renegotiation
* schannel: renegotiating SSL/TLS connection
* schannel: SSL/TLS connection renegotiated
< HTTP/1.1 400 Bad Request
< x-frame-options: SAMEORIGIN
< access-control-allow-origin: *
< access-control-allow-headers: accept, apiKey, content-type, origin, x-requested-with
< access-control-allow-methods: GET, HEAD, OPTIONS
< access-control-allow-credentials: false
< date: Wed, 10 Jul 2024 03:43:12 GMT
< content-length: 0
< apikey: Yes
< strict-transport-security: max-age=31536000
<
* Connection #0 to host services.nvd.nist.gov left intact

[chadlwilson]

@phs-glenfordham I believe it usually means there is a problem with your API key, or it is getting corrupted/lost somehow between how you supply it and the library at the other end. The change really just improves the logging so you can tell generally what was supplied on the API call, so you can figure out if you supplied it correctly to the CLI, Maven, Gradle plugin etc.

As for your curl issue, likely the way the URL is being escaped isn't appropriate for Windows/CMD. With the exact same command on MacOS I get 404 message: Invalid apiKey as expected.

You'll probably need to quote the URL properly, not rely on backslash escaping:

curl -H "Accept: application/json" -H "apiKey: redacted" -v "https://services.nvd.nist.gov/rest/json/cves/2.0?cpeName=cpe:2.3:o:microsoft:windows_10:1607:*:*:*:*:*:*:*"

[phs-glenfordham]

Thanks for the quick response Jeremy, and apologies for hijacking the thread. Yes, it makes sense that cmd is messing with the input, thanks for the thought.

The peace of mind that the key is working was helpful, and we may have solved our issue with the 403/404 error when using the gradle plugin now. The new release will be useful if we ever need to implement it for another project though, still keen :)

[chadlwilson]

@phs-glenfordham Would you mind sharing what your issue was getting the API key through the Gradle Plugin correctly? Perhaps we can do something to give faster feedback within the Gradle plugin itself that something looks wrong in some cases. I commonly see problems when people are using Gradle project properties or system properties and trying to supply them from outside, and issues with multi-project Gradle setups.

[zsbee]

FYI, we had the same issue, and we have 2 api keys to NVD and both reported valid jsons. Even tried running it without an NVD API Key (that warns us that it will be slow) and that was also throwing errors.

upgrading to 10.0.2 from 10.0.1 fixed it for us.

[chadlwilson]

Yes, the upgrade to 10.0.2+ is mandatory and the source of most people's problems as NVD are blocking earlier (or unidentified) versions of ODC due to mitigating retry DDoS (mainly from 9.x versions).

But some folks are also having to supply API keys for the first time, as they are moving from 8.x to using the API, OR trying to reduce the chance of getting throttled given the load the API is under. Some of those are probably having issues supplying their keys properly through the plugins/CLI to the library.

Curl will tell you your API key is valid, but not that it is being picked up by ODC correctly.

[phs-glenfordham]

@phs-glenfordham Would you mind sharing what your issue was getting the API key through the Gradle Plugin correctly? Perhaps we can do something to give faster feedback within the Gradle plugin itself that something looks wrong in some cases. I commonly see problems when people are using Gradle project properties or system properties and trying to supply them from outside, and issues with multi-project Gradle setups.

we were using a custom environment variable to load it into the gradle plugin, something like this

dependencyCheck {
    nvd {
        apiKey = System.getenv("NVD_API_KEY")
    }
}

The issue was simply that the environment variable value we needed to use hadn't refreshed properly yet, because we had re-requested one several times trying to make sense of the issue before upgrading. The issue we had with curl didn't help, as then we thought it was just a general issue. I think it would certainly help if the contents of the NVD request could be optionally dumped to the console, especially if gradle is running in debug mode - would've realised our stupidity sooner :)

[chadlwilson]

Here's my summary given the current load. Hope it helps.

• >= 9.x, <= 10.0.1: Will get 403/404 due to NVD rejecting old clients #6817
  1. Upgrade to 10.0.2.
• >= 10.0.2 Without API key: Quite likely to be getting 503s due to load per #6758
  1. Consider getting an API key. :-)
• >= 10.0.2 With valid API key: Should be working, but
  • If getting 503: The retries should get you through eventually as load should be improving.
    1. But you might want to look at https://jeremylong.github.io/DependencyCheck/data/cachenvd.html or https://jeremylong.github.io/DependencyCheck/data/cacheh2.html to reduce your dependency on the NVD by reducing # of calls.
    2. Consider adjusting the ODC retry delay to be longer.
  • if getting 403/404:
    1. Double check your API key is valid using curl or similar.
    2. Double check the same API key is going through correctly from your plugin/CLI. The logging @jeremylong is adding may help improve this, but if using Gradle you can try logging it temporarily within Gradle to make sure it is being read correctly from your environment.
       • If you have been re-generating API keys, check you are using the correct one. Old keys are invalidated when you regenerate from the same email address.
    3. Turn on Maven debug logging, Gradle --info logging, or gradle --stacktrace and see if there is some other connectivity issue to the NVD API other than a 403/404/503. (especially if you have recently moved from ODC 8.x)

[chadlwilson]

The issue was simply that the environment variable value we needed to use hadn't refreshed properly yet, because we had re-requested one several times trying to make sense of the issue before upgrading.

Ahh OK, yes, the invalidation of the old keys when re-generating new keys is a real pain, as there is no distinction between the two types of failures on NVD and if working in a shared environment where multiple people have access to regenerate the key you can end up in a really confusing mess. Let me add that to my summary.

[paksydavid]

On our case upgrading to 10.0.2 conmpletelysolved this issue. Many thanks @jeremylong :pray:

[1-mrudulaahire]

@phs-glenfordham I believe it usually means there is a problem with your API key, or it is getting corrupted/lost somehow between how you supply it and the library at the other end. The change really just improves the logging so you can tell generally what was supplied on the API call, so you can figure out if you supplied it correctly to the CLI, Maven, Gradle plugin etc.

As for your curl issue, likely the way the URL is being escaped isn't appropriate for Windows/CMD. With the exact same command on MacOS I get 404 message: Invalid apiKey as expected.

You'll probably need to quote the URL properly, not rely on backslash escaping:

curl -H "Accept: application/json" -H "apiKey: redacted" -v "https://services.nvd.nist.gov/rest/json/cves/2.0?cpeName=cpe:2.3:o:microsoft:windows_10:1607:*:*:*:*:*:*:*"

Thanks for curl commad it's help in knowing about api key validation. It always showing as Invalid. Do I need to wait for some to get activated or what once i get mail to check key validation

[chadlwilson]

@1-mrudulaahire If it has been some time since you activated (hours), I would try again with a new key. Be careful not to double-click or refresh pages when you activate the key, request new keys etc; or you can end up overwriting your own key and getting confused about which one is valid.

Other than that we can't really help you as this is the American NIST NVD's process/system and I don't believe they publish openly how long it takes for an activated key to be usable within the systems (might be immediate? might not?)

This is all documented at https://nvd.nist.gov/developers/start-here

Each API Key is associated with a single email address. If an email address is used to request an additional API key, clicking the single-use hyperlink will invalidate the key previously associated with that email address. The key will not be invalidated if the email address is used to request another key, but the hyperlink is not opened. There is no process for retrieving a forgotten key or confirming whether a key has been requested or activated by any email address.