I have, like most others, experienced the dreaded fact that downloading the new cve files, even with an NVS API key it is basically impossible to make it work right now.
So essentially all my pipelines are failing on OWASP analysis.
I figured id do some investgation and discovered an alternative solution where you have the CVE cached on you own storage.
Use Parameter: --nvdDatafeed
And by using the tooling https://github.com/jeremylong/Open-Vulnerability-Project/tree/main/vulnz#caching-the-nvd-cve-data
I have successfully downloaded a local version of the CVE cache.
My actual command to download the data. Remember to set the NVS key as a variable prior.
java -jar vulnz-5.0.2.jar cve --cache --directory cache --delay 6000 --threads 6 --maxretry 40 --debug
That turn into approx 22 files ranging from 600 KB to 13 Mb.
Problem Description.
I have setup my Azure Devops Pipeline to download NVD cache from custom storage.
It download most files but then throws - Out of memory Exception.
Caused by: java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: Java heap space
Then the analysis step fails.
Expectation.
Using a local cached version of the NVD files or downloading them from the standard source should not cause a Java Heap - Out Of Memory exception.
I would expect that since i actually have access to the files that the OWASP analysis would finally work again.
What can I do to remedy this?
I have attached a slightly redacted log from the pipeline for error analysis.
Please let me know if i can do anything to assist.
azuredevops-owasp-logs-redacted.log
ejohn20
commented
7 months ago
A bit of context.
I have, like most others, experienced the dreaded fact that downloading the new cve files, even with an NVS API key it is basically impossible to make it work right now. So essentially all my pipelines are failing on OWASP analysis.
I figured id do some investgation and discovered an alternative solution where you have the CVE cached on you own storage.
Use Parameter: --nvdDatafeed And by using the tooling https://github.com/jeremylong/Open-Vulnerability-Project/tree/main/vulnz#caching-the-nvd-cve-data I have successfully downloaded a local version of the CVE cache. My actual command to download the data. Remember to set the NVS key as a variable prior.
java -jar vulnz-5.0.2.jar cve --cache --directory cache --delay 6000 --threads 6 --maxretry 40 --debugThat turn into approx 22 files ranging from 600 KB to 13 Mb.
Problem Description.
I have setup my Azure Devops Pipeline to download NVD cache from custom storage. It download most files but then throws - Out of memory Exception.
Caused by: java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: Java heap spaceThen the analysis step fails.Expectation.
Using a local cached version of the NVD files or downloading them from the standard source should not cause a Java Heap - Out Of Memory exception. I would expect that since i actually have access to the files that the OWASP analysis would finally work again.
What can I do to remedy this?
I have attached a slightly redacted log from the pipeline for error analysis. Please let me know if i can do anything to assist. azuredevops-owasp-logs-redacted.log