MavenCentral (504): Gateway Time-out

[TheRealArlie]

Describe the bug While trying to scan a *.jar file using dependency-check in the most recent version (8.0.1) it tries to reach https://search.maven.org/solrsearch/select and gets a gateway timeout.

Version of dependency-check used 8.0.1 in the official docker image

Log file error.log

DEBUG - Searching Central url https://search.maven.org/solrsearch/select?q=1:2973d150c0dc1fefe998f834810d68f278ea58ec&wt=xml
2023-01-18 12:47:01,395 org.owasp.dependencycheck.analyzer.CentralAnalyzer:346
DEBUG - Could not connect to Central search (tries left: 6): Could not connect to MavenCentral (504): Gateway Time-out

To Reproduce

1. Download https://github.com/WebGoat/WebGoat/releases/download/7.1/webgoat-container-7.1-exec.jar
2. run /usr/share/dependency-check/bin/dependency-check.sh -s webgoat-container-7.1-exec.jar -l /dev/stdout

Expected behavior It should finish the scan within some seconds or minutes.

[TheRealArlie]

Thanks for the fast answer. Do you by chance know how to configure it? Is there some documentation for this?

[aikebah]

https://jeremylong.github.io/DependencyCheck/ look for the option to disable the CentralAnalyzer in your tool. For the docker image that's the configuration options for the CLI

[TheRealArlie]

Thanks for the fast reply. Can I get two more questions off my chest?

I can't find anything about this in the documentary. Am I correct in assuming that the central analyzer is the problem and I should disable it? The only thing I can find in the CLI overview is this:

--disableCentral       

Sets whether the Central Analyzer will be used. Disabling this analyzer is not recommended as it could lead to false negatives (e.g. libraries that have vulnerabilities may not be reported correctly). If this analyzer is being disabled there is a good chance you also want to disable the Artifactory or Nexus Analyzer.

If I understand correctly, this is not recommended. Could you help me with your better understanding of the ODC architecture on this? What is the central analyzer doing?

The second question would be: You have changed the ticket from bug to question. Is the issue only with me or is it reproducible by you? If it also occurs with you, it could still be a bug, correct? If not, then it could be due to our corporate network.

Thanks for all your efforts !!!

[rseeton]

@TheRealArlie - I am encountering the same issue. However, it appears the problem is with the Maven site: ( https://github.com/jeremylong/DependencyCheck/issues/5341 ).

[aikebah]

@TheRealArlie changed it from bug to question, because there is not a bug in the application, but the application is suffering from the current availability issues of Maven Central's search API. To my knowledge this is the first serious disturbance in Maven Central's search availability and I hope they'll be able to pinpoint the root-cause and put a resolution in place soon.

The reason that it is not recommended to disable it is that the DependencyCheck CLI has only limited information to go by (filename and some metadata in the zip). The chances of avoiding false-positive noise are significantly increased for libraries that are published in Maven Central (for which the pom.xml can be retrieved) as they allow to use the Maven coordinates (groupId, artifactId, version) in the evaluation of what libraries was scanned by DependencyCheck.

Disabling the CentralAnalyzer will increase the chance that libraries are matched to an incorrect 'CPE' (Common Product Enumeration) leading to vulnerabilities being reported that involve a different library (False positives). The CPE is the identifier for libraries in the NIST NVD (nvd.nist.gov) to link vulnerabilities(CVEs) to the vulnerable library versions.

For the time being disabling Central Analyzer seems the most workable approach. When Maven Central Search is back to stable you could consider re-enabling it to benefit from fewer false-positives.

[TheRealArlie]

Thank you a lot for the detailed answers!!!

Please decide on your own whether the ticket should be closed or left open till Maven Central has fixed its problems and others encountering the same bug can find this here and know that they have to wait.

Cheers, Robin

[hariom282538]

getting 504

`parts [ 'org.awaitility', 'awaitility', 'jar', '4.0.3', 'compile' ] Note: Unnecessary use of -X or --request, GET is already inferred. % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 Trying 3.227.172.120:443... 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 Connected to search.maven.org (3.227.172.120) port 443 (#0)

• ALPN, offering h2
• ALPN, offering http/1.1
• successfully set certificate verify locations:
• CAfile: /etc/ssl/cert.pem
• CApath: none
• (304) (OUT), TLS handshake, Client hello (1): } [321 bytes data]
• (304) (IN), TLS handshake, Server hello (2): { [98 bytes data]
• TLSv1.2 (IN), TLS handshake, Certificate (11): { [4945 bytes data]
• TLSv1.2 (IN), TLS handshake, Server key exchange (12): { [333 bytes data]
• TLSv1.2 (IN), TLS handshake, Server finished (14): { [4 bytes data]
• TLSv1.2 (OUT), TLS handshake, Client key exchange (16): } [70 bytes data]
• TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1): } [1 bytes data]
• TLSv1.2 (OUT), TLS handshake, Finished (20): } [16 bytes data]
• TLSv1.2 (IN), TLS change cipher, Change cipher spec (1): { [1 bytes data]
• TLSv1.2 (IN), TLS handshake, Finished (20): { [16 bytes data]
• SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
• ALPN, server accepted to use h2
• Server certificate:
• subject: CN=maven.org
• start date: Jan 6 00:00:00 2023 GMT
• expire date: Feb 4 23:59:59 2024 GMT
• issuer: C=US; O=Amazon; CN=Amazon RSA 2048 M02
• SSL certificate verify ok.
• Using HTTP2, server supports multiplexing
• Connection state changed (HTTP/2 confirmed)
• Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
• Using Stream ID: 1 (easy handle 0x155812e00)

  GET /solrsearch/select?q=g:org.awaitility%20AND%20a:awaitility&rows=30&wt=json HTTP/2 Host: search.maven.org user-agent: curl/7.79.1 accept: /

• Connection state changed (MAX_CONCURRENT_STREAMS == 128)! 0 0 0 0 0 0 0 0 --:--:-- 0:01:00 --:--:-- 0< HTTP/2 504 < date: Mon, 30 Jan 2023 15:57:11 GMT < content-type: text/html < content-length: 160 < server: nginx < { [160 bytes data] 100 160 100 160 0 0 2 0 0:01:20 0:01:00 0:00:20 40
• Connection #0 to host search.maven.org left intact`

[praveenkumarp893]

@TheRealArlie changed it from bug to question, because there is not a bug in the application, but the application is suffering from the current availability issues of Maven Central's search API. To my knowledge this is the first serious disturbance in Maven Central's search availability and I hope they'll be able to pinpoint the root-cause and put a resolution in place soon.

The reason that it is not recommended to disable it is that the DependencyCheck CLI has only limited information to go by (filename and some metadata in the zip). The chances of avoiding false-positive noise are significantly increased for libraries that are published in Maven Central (for which the pom.xml can be retrieved) as they allow to use the Maven coordinates (groupId, artifactId, version) in the evaluation of what libraries was scanned by DependencyCheck.

Disabling the CentralAnalyzer will increase the chance that libraries are matched to an incorrect 'CPE' (Common Product Enumeration) leading to vulnerabilities being reported that involve a different library (False positives). The CPE is the identifier for libraries in the NIST NVD (nvd.nist.gov) to link vulnerabilities(CVEs) to the vulnerable library versions.

For the time being disabling Central Analyzer seems the most workable approach. When Maven Central Search is back to stable you could consider re-enabling it to benefit from fewer false-positives.

Hi @aikebah Thank you for the information. We have currently disabled the Central Analyzer. When can we expect the Maven Central Search back to stable state ?

[aikebah]

@praveenkumarp893 You'd have to check with the people in charge of maven central.

Overall it appears to mostly be available with some intermittent failures.

https://status.maven.org/

[praveenkumarp893]

Okay Thank you @aikebah

[Michenux]

It happens quite frequently. No fallback server solution ?

[aikebah]

@Michenux

If you have an Artifactory server that is also proxying Maven Central you could use the ArtifactoryAnalyzer and switch off the CentralAnalyzer.

If you have a Nexus v2 server also proxying Maven Cental you could use the NexusAnalyzer (known to work with both Pro and community version of v2) and switch off the CentralAnalyzer.

If you happen to have a Nexus pro server we would be happy to receive a PR for a working Nexus v3 Analyzer. NexusAnalyzer is coded for v2 and fails to work on v3 as the API has been rewritten in V3, see #5319)

Also note that CentralAnalyzer employs results-caching by default, so ensuring that you share the datafolder of dependencyCheck on your CI-infrastructure should reduce the chances of running into a 504 case, both by cache reuse avoiding the API call (which means it will not run into 504) and by reducing the overall stress on the Central search API increasing the chances for successful use of the API for uncached artifact search results.

[Michenux]

We use Nexus 3 community :( I will look further how to implement the ci cache to mitigate the issue. As we use a postgres database for dependency-check, i thought it was not really useful to put the data directory in cache.

[aikebah]

We use Nexus 3 community :( I will look further how to implement the ci cache to mitigate the issue. As we use a postgres database for dependency-check, i thought it was not really useful to put the data directory in cache.

Your remark got me to revisit Sonatype's pages and looks like the search APIs allowing search by SHA are also available in Nexus 3 community. I intend to give it a spin on a local VM and see whether that's correct and whether I can manage to build a Nexusv3Analyzer for it.

In any case the sharing of data directory is useful. Besides the CentralAnalyzer caches there's also caching of the OSSIndexAnalyzer which can be beneficial to avoid hitting the OSSIndex rate-limits (unauthenticated is very restrictive in the limit, registered account might hit the higher rate-limits if you use it massively and don't cache - https://ossindex.sonatype.org/doc/rest), NodeAudit results, RetireJS and the hostedSuppressions file.

[ImSaSu]

We are also running into these issues (just happened again, today). Am I correct in surmising that the API from maven central might be getting deprecated in the future, now that they have marked it as 'legacy', and should this project switch to their new implementation early as a result? Especially due to the reoccurring issues.

It would be great to see this feature in the future (possibly still using both APIs, or one as a fallback mechanism of the other). Even better if it is more configurable than currently. How did your investigation into the matter go, @aikebah ? Your efforts are much appreciated!

The documentation does not say and I am much to unknowing about the subject, but is this API something a local nexus mirror can provide? If so, at which subdomain, as the one mentioned in the opening post does not respond on our local nexus instance. Although possibly we are just missing some required configuration?

We will try to proceed with the mentioned workaround, and reuse the data folder.

[giacgbj]

It would be nice to have a parameter similar to "ossIndexRemoteErrorWarnOnly" (e.g., "mavenCentralRemoteErrorWarnOnly"), which can be set to true to avoid failing all the build due to this error.

[aikebah]

It would be great to see this feature in the future (possibly still using both APIs, or one as a fallback mechanism of the other). Even better if it is more configurable than currently. How did your investigation into the matter go, @aikebah ? Your efforts are much appreciated!

Initial efforts have started, hope to spend some serious time to finish up the changes from next Thursday onward and expect that to result in a PR to add Nexusv3 as an alternative flavour of Sonatype Nexus support. At least the initial experiments make me optimistic in achieving a satisfiable result.

Regarding central search: the 'new API' does not have a public search API. The 'new central search' website uses private internal APIs.

[aikebah]

@ImSaSu PR #5849 is out now for supporting Nexus V3 in the NexusAnalyzer