[SpringCloudGateway]: Netty HttpClient failed to connect with SSLHandshakeException after establishing HTTPS connection with endpoint

[deepak-chhetri]

We have built API Gateway application using Spring Cloud Gateway 4.0 (on webflux) framework and Azul Zulu OpendJDK 17. The application is able to intercept incoming request and redirect it to the configured downstream HTTPS endpoint using TLS certificate. However, there is one request in which the gateway app unable to establish HTTPS connection with configured downstream endpoint and getting below SSL Handshake error (soon after connection and active channel):

io.netty.handler.codec.DecoderException: javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure

We have configured the TLS certificate for routing request to endpoint in following property which is passed as JVM_ARGS in java command line:

javax.net.ssl.trustStore=<path to .pkcs12 keystore>

We enabled more granular logs from Netty library by setting reactor.netty=DEBUG level and found below lines after above exception:

HttpServerOperations | [d7cffb05-1, L:/<Source IP:Port> - R:/<Destination IP:Port>] Detected non persistent http connection, preparing to close
HttpServerOperations | [d7cffb05-1, L:/<Source IP:Port> - R:/<Destination IP:Port>] Last HTTP response frame
HttpServerOperations | [d7cffb05-1, L:/<Source IP:Port> - R:/<Destination IP:Port>] Last HTTP packet was sent, terminating the channel

However, Netflix Zuul Gateway application (running on same server in which Spring Cloud Gateway app is running) is able to connect with the same HTTPS endpoint and redirect the request to it successfully using same TLS certificate. Note: Netflix Zuul is using RestTemplate to connect whereas Spring Cloud Gateway is using Netty HttpClient.

We have tried many combination with different properties however still stuck with same error. How can we resolve this SSL handshake failure soon after connection is established? Any help is highly appreciated in this direction.

[lhotari]

answered in https://github.com/netty/netty/issues/14087#issuecomment-2140991370

[deepak-chhetri]

@lhotari We checked the remote IBM server/endpoint and found following error in server logs:

SSL0222W: SSL Handshake Failed, No ciphers specified (no shared ciphers or no shared protocols).

It seems that cipher configuration is missing in request sent to endpoint server from Spring Cloud Gateway HttpClient. Could you please share details around how to configure cipher suites in Spring Cloud Gateway HttpClient?

We have used following command to list down the ciphers used by endpoint server as follows:

openssl s_client -connect serverAddress:port

and got following output with cipher details:

SSL-Session:
    Protocol: TLSv1.2
    Cipher: AES256-SHA256
    Verify return code: 0 (ok)

[lhotari]

@deepak-chhetri to investigate the problem and suggest a proper solution, it could be helpful to also share the output of sslscan (https://github.com/rbsec/sslscan) for the target server. sslscan is available in package managers such as homebrew (brew install sslscan) or ubuntu apt (sudo apt-get install sslscan).

[deepak-chhetri]

@lhotari Following is the relevant output of sslscan utility:

SSL/TLS Protocols:
SSLv2     disabled
SSLv3     disabled
TLSv1.0  disabled
TLSv1.1  disabled
TLSv1.2  enabled
TLSv1.3  disabled

TLS Fallback SCSV:
Server supports TLS Fallback SCSV

TLS renegotiation:
Session renegotiation not supported

TLS Compression:
Compression disabled

Supported Server Cipher(s):
Preferred  TLSv1.2   256 bits  AES256-SHA256
Accepted  TLSv1.2   128 bits  AES128-SHA256
Accepted  TLSv1.2   128 bits  DES-CBC3-SHA

[deepak-chhetri]

@lhotari Could you please share HttpClient settings to configure cipher suites to connect with endpoint server over HTTPS?

[lhotari]

@deepak-chhetri I don't have a ready made example. I guess you should be able to customize that by adding a bean that implements org.springframework.cloud.gateway.config.HttpClientCustomizer.

[lhotari]

You might also be able to override httpClientSslConfigurer bean with your own implementation that configures the ciphers. https://github.com/spring-cloud/spring-cloud-gateway/blob/920b3a81abb350afa9f53ad6be705560219cdfcb/spring-cloud-gateway-server/src/main/java/org/springframework/cloud/gateway/config/GatewayAutoConfiguration.java#L756-L761

[deepak-chhetri]

@lhotari Is there any settings that can be configured in yml file directly like spring.cloud.gateway.httpclient.ssl ?

[lhotari]

@lhotari Is there any settings that can be configured in yml file directly like spring.cloud.gateway.httpclient.ssl ?

No. That's the reason why you need either of the solutions that I mentioned in the previous comments.

[lhotari]

@deepak-chhetri Although it is possible that defaults for the ciphers could be configured with a system property.

[deepak-chhetri]

@lhotari Could you please share example to configure default cipher suite?

[lhotari]

@deepak-chhetri Unfortunately not.

[lhotari]

It's also worth checking if "Configuring default extensions" is related.

Some TLS implementations may not handle unknown extensions properly. As a result, you might encounter unexpected interoperability issues when the JDK introduces new extensions.

(check https://bugs.openjdk.org/browse/JDK-8217633 for more details)

Some servers get confused by TLS handshake extensions that clients send and report it as a cipher error (based on comment https://bugs.openjdk.org/browse/JDK-8257825).

You could try passing -Djdk.tls.client.disableExtensions=supported_versions to see if that helps, unless configuring ciphers is possible or doesn't fix the issue.