search

fasten-project / vulnerability-producer

Gathers, enriches and publishes vulnerability information to a Kafka topic.
https://www.fasten-project.eu/
Apache License 2.0
6 stars 3 forks source link

Producer crashes when NIST cannot be reached #85

Open MagielBruntink opened 3 years ago

MagielBruntink commented 3 years ago

The producer crashes and restarts if its NIST NVD downloads don't work, which happens now-and-then due to internet issues beyond our control. It would be better if the producer didn't crash though :-)

Example:

[2021-06-14 11:16:42,710] [DEBUG] [main] [o.o.d.u.SSLSocketFactoryEx] - TLSv1.3
org.owasp.dependencycheck.utils.DownloadFailedException: Download failed, unable to copy 'https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-2002.json.gz' to '/mnt/fasten/vuln/producer/nvd/nvdcve-1.1-2002.json.gz'
    at org.owasp.dependencycheck.utils.Downloader.fetchFile(Downloader.java:98)
    at org.owasp.dependencycheck.utils.Downloader.fetchFile(Downloader.java:74)
    at eu.fasten.vulnerabilityproducer.utils.parsers.NVDParser.downloadCVEs(NVDParser.java:115)
    at eu.fasten.vulnerabilityproducer.utils.parsers.NVDParser.getVulnerabilities(NVDParser.java:226)
    at eu.fasten.vulnerabilityproducer.utils.parsers.ParserManager.getVulnerabilitiesFromParsers(ParserManager.java:95)
    at eu.fasten.vulnerabilityproducer.VulnerabilityProducer.start(VulnerabilityProducer.java:111)
    at eu.fasten.vulnerabilityproducer.Main.run(Main.java:134)
    at picocli.CommandLine.executeUserObject(CommandLine.java:1729)
    at picocli.CommandLine.access$900(CommandLine.java:145)
    at picocli.CommandLine$RunLast.handle(CommandLine.java:2101)
    at picocli.CommandLine$RunLast.handle(CommandLine.java:2068)
    at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:1935)
    at picocli.CommandLine.execute(CommandLine.java:1864)
    at eu.fasten.vulnerabilityproducer.Main.main(Main.java:82)
Caused by: org.owasp.dependencycheck.utils.DownloadFailedException: Error downloading file https://nvd.nist.gov/feeds/json/cve/1.1/nvdcve-1.1-2002.json.gz; unable to connect.
    at org.owasp.dependencycheck.utils.HttpResourceConnection.obtainConnection(HttpResourceConnection.java:238)
    at org.owasp.dependencycheck.utils.HttpResourceConnection.fetch(HttpResourceConnection.java:138)
    at org.owasp.dependencycheck.utils.Downloader.fetchFile(Downloader.java:94)
    ... 13 more
Caused by: java.net.UnknownHostException: nvd.nist.gov
    at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:220)
    at java.base/java.net.Socket.connect(Socket.java:609)
    at java.base/sun.net.NetworkClient.doConnect(NetworkClient.java:177)
    at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:474)
    at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:569)
    at java.base/sun.net.www.protocol.https.HttpsClient.<init>(HttpsClient.java:265)
    at java.base/sun.net.www.protocol.https.HttpsClient.New(HttpsClient.java:372)
    at java.base/sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(AbstractDelegateHttpsURLConnection.java:203)
    at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1187)
    at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1081)
    at java.base/sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:189)
    at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:168)
    at org.owasp.dependencycheck.utils.HttpResourceConnection.obtainConnection(HttpResourceConnection.java:178)
    ... 15 more
java.lang.NullPointerException
    at eu.fasten.vulnerabilityproducer.utils.parsers.NVDParser.getVulnerabilities(NVDParser.java:231)
    at eu.fasten.vulnerabilityproducer.utils.parsers.ParserManager.getVulnerabilitiesFromParsers(ParserManager.java:95)
    at eu.fasten.vulnerabilityproducer.VulnerabilityProducer.start(VulnerabilityProducer.java:111)
    at eu.fasten.vulnerabilityproducer.Main.run(Main.java:134)
    at picocli.CommandLine.executeUserObject(CommandLine.java:1729)
    at picocli.CommandLine.access$900(CommandLine.java:145)
    at picocli.CommandLine$RunLast.handle(CommandLine.java:2101)
    at picocli.CommandLine$RunLast.handle(CommandLine.java:2068)
    at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:1935)
    at picocli.CommandLine.execute(CommandLine.java:1864)
    at eu.fasten.vulnerabilityproducer.Main.main(Main.java:82)