openssl (1.1.1) picking the incorrect cipher (i.e. TLS1.3 cipher-TLS_AES_256_GCM_SHA384) for the TLS1.2 client hello and this causing TLS session failing

[ganga1980]

root@omsagent-rs-65cbfcbd86-fnqhw:/# ruby -v ruby 2.6.6p146 (2020-03-31 revision 67876) [x86_64-linux-gnu] root@omsagent-rs-65cbfcbd86-fnqhw:/# openssl OpenSSL> version OpenSSL 1.1.1 11 Sep 2018

Because of this, server asking to change cipherspec which in turn causing fatla error unexpected message(10).

is this known issue with the 1.1.1 version?

[rhenium]

TLS 1.3 doesn't use the version field for the version negotiation and instead a TLS 1.3-capable ClientHello contains a "supported_versions" extension for that purpose.

A TLS <= 1.2 server should be ignoring any unknown cipher suites or extensions sent by the client. I don't think that is triggering the unexpected_message alert.

[ganga1980]

Hi, @rhenium, why the TLS 1.2-capable Client Hello sending supported versions with TLS 1.3 and ciphers of TLS 1.3? Either client has to send TLS 1.3 -capable Client Hello? or not send the TLS 1.3 and cipher in supported version and cipher? I think, this is clearly bug in the openssl.

[rhenium]

Please provide the steps/code snippet to reproduce the issue.

I don't find anything irregular in the ClientHello; it looks like a typical client that supports TLS 1.0-1.3.

[ganga1980]

Please provide the steps/code snippet to reproduce the issue.

I don't find anything irregular in the ClientHello; it looks like a typical client that supports TLS 1.0-1.3.

Hi, @rhenium, why the client sending TLS 1.2- capable client hello when it has capability to support for TLS 1.3? Isnt this issue at the client side?

[rhenium]

Please see RFC 8446 (https://datatracker.ietf.org/doc/html/rfc8446#appendix-D) for how protocol version negotiation works.

[ganga1980]

Hi, @rhenium,
Here is the sample code how we create HTTP client to talk to kubernetes server endpoint.

 Net::HTTP.start(uri.host, uri.port, :use_ssl => true, :ca_file => @@CaFile, :verify_mode => OpenSSL::SSL::VERIFY_PEER, :open_timeout => 20, :read_timeout => 40) do |http|
              kubeApiRequest = Net::HTTP::Get.new(uri.request_uri)
              kubeApiRequest["Authorization"] = "Bearer " + getTokenStr
              @Log.info "KubernetesAPIClient::getKubeResourceInfo : Making request to #{uri.request_uri} @ #{Time.now.utc.iso8601}"
              response = http.request(kubeApiRequest)
              @Log.info "KubernetesAPIClient::getKubeResourceInfo : Got response of #{response.code} for #{uri.request_uri} @ #{Time.now.utc.iso8601}"
            end

https://github.com/microsoft/Docker-Provider/blob/ci_prod/source/plugins/ruby/KubernetesApiClient.rb#L57

[ganga1980]

Hi, @rhenium, Here is the sample code how we create HTTP client to talk to kubernetes server endpoint.

 Net::HTTP.start(uri.host, uri.port, :use_ssl => true, :ca_file => @@CaFile, :verify_mode => OpenSSL::SSL::VERIFY_PEER, :open_timeout => 20, :read_timeout => 40) do |http|
              kubeApiRequest = Net::HTTP::Get.new(uri.request_uri)
              kubeApiRequest["Authorization"] = "Bearer " + getTokenStr
              @Log.info "KubernetesAPIClient::getKubeResourceInfo : Making request to #{uri.request_uri} @ #{Time.now.utc.iso8601}"
              response = http.request(kubeApiRequest)
              @Log.info "KubernetesAPIClient::getKubeResourceInfo : Got response of #{response.code} for #{uri.request_uri} @ #{Time.now.utc.iso8601}"
            end

https://github.com/microsoft/Docker-Provider/blob/ci_prod/source/plugins/ruby/KubernetesApiClient.rb#L57

[rhenium]

It looks like the basic usage. Can openssl s_client -connect {host}:{port} (or openssl s_client -tls1_2 to disable all TLS 1.3 support) connect to the server?

[ganga1980]

It looks like the basic usage. Can openssl s_client -connect {host}:{port} (or openssl s_client -tls1_2 to disable all TLS 1.3 support) connect to the server?

Here is the o/p of the commands which you have asked for

openssl s_client -connect host:port

---

New, TLSv1.3, Cipher is TLS_AES_256_GCM_SHA384 Server public key is 4096 bit Secure Renegotiation IS NOT supported Compression: NONE Expansion: NONE No ALPN negotiated Early data was not sent

openssl s_client -connect host:port -tls1_2

New, TLSv1.2, Cipher is ECDHE-RSA-AES256-GCM-SHA384 Server public key is 4096 bit Secure Renegotiation IS supported Compression: NONE Expansion: NONE No ALPN negotiated

If we specify :ssl_version => TLSv1_2 for HTTP.start , then everything works. As I mentioned, looks like issue with negotiating with TLSv1.3 with the cipher - TLS_AES_256_GCM_SHA384

[rhenium]

This is strange. openssl s_client can connect with TLS 1.3 but Ruby/OpenSSL cannot even though they should be doing basically the same thing.

Unfortunately, as I won't have access to the private Kubernetes API server, I'm afraid there is not much we can do to help here.

[ganga1980]

Hi, @rhenium, I am suspecting this could be issue with Server side. Is it Change Cipher Spec from Server as response to Client Hello is not an issue? If you observe the protocal interaction betweenc client and server, client initiated with TLS1.2 - capable Client Hello and server responded with Change Cipher Spec, then client saying this is in unexpected . From this interaction, Change Cipher Spec is not suppose to be send the server, right?

[ganga1980]

Hi, @rhenium , Can you help me to understand how openssl determines whether to initiate TLS 1.2 capable client hello or TLS 1.3 capable client hello? We are using the ruby within docker container, some cases, ruby client initiates TLS 1.2 capable client hello where as other case it initiates TLS 1.3 capable client hello? will there any negotiation happens between client and server to determine whether TLS 1.2 capable client hello or TLS 1.3 client hello?

[rhenium]

ChangeCipherSpec can be sent by the server in both protocol versions.

TLS is a client initiated protocol, so a client doesn't have knowledge of the supported protocol versions by the server. The ClientHello in your screenshots looked compatible with TLS 1.0-1.3. The server should decide which one among these to use.

[ganga1980]

Hi, @rhenium, I am seeing the weird behavior in the client side regarding TLS 1.2 Record Layer Client Hello vs TLS 1.3 Record Client hello. With the same HTTP client (ruby 2.6.6p146 & openssl version 1.1.1) within the same container, on certain nodes, client initiates TLS 1.2 Record Layer Client Hello but where as the other nodes, same container (meaning Ruby HTTP & openssl client) initiates the TLS 1.3 Record Layer Client Hello message. No issues with regards the TLS session establishment from the nodes where the TLS 1.3 Record Layer Client Hello message but the TLS session failed with unexpected message(10) alert message where the TLS 1.2 client hello message initiated. Both the scenarios, supported versions in the TLS client hello are TLS 1.3, TLS 1.2, TLS 1.1 & TLS 1.0. So question, how the Open SSL determines whether to send TLS 1.2 Record Layer Client hello vs TLS 1.3 Record Layer Client Hello?

[ganga1980]

Hi, @nurse , I am suspecting the issue I am seeing is related to this change - https://github.com/ruby/ruby/commit/dcea9198a9d80bdf4eeacd9d9e9d883850a4a8d2#diff-a1d29a94def02829fd4f9ba591199acf079e028f5a2002a77c363eb01212e112 . do you have any insights why the same ruby client code behaving differently with TLS 1.2 client hello vs TLS 1.3 client hello? is it something with min & max version?

[DavidMealha]

I can confirm that this is still an issue, while using OpenSSL 1.1.1 and calling a server that uses a TLSv1.3 cipher suite (in my case AES128-GCM-SHA256) the connection is reset. If we set the version to TLSv1.2 everything works fines.

Using openssl s_client with -tls1_3 or -tls1_2 works as expected, as well.