TLS1.3 with brainpool version incompatibility

[bernd-edlinger]

When an openssl-3.0 client wants to support brainpool server certificates, but also allow TLS1.3 with RSA certificates, everything works fine, when both client and server enable brainpool groups, in addition to nist groups. So the client has to use this command line: ./openssl s_client -groups P-256:brainpoolP256r1 and the openssl-3.0 server uses this command line for brainpool: ./openssl s_server -cert ../test/certs/server-ecdsa-brainpoolP256r1-cert.pem -key ../test/certs/server-ecdsa-brainpoolP256r1-key.pem -groups P-256:brainpoolP256r1 or this command line for RSA: ./openssl s_server everything works fine, RSA use TLS1.3 and ECC use TLS1.2. Also if both server and client use openssl-3.2 everything works as well, except the ECC use TLS1.3, as expected. BUT the openssl-3.0 client cannot connect to the openssl-3.2 server when the server uses brainpool:

$ ./openssl s_client -groups P-256:brainpoolP256r1 -trace
CONNECTED(00000003)
Sent Record
Header:
  Version = TLS 1.0 (0x301)
  Content Type = Handshake (22)
  Length = 327
    ClientHello, Length=323
      client_version=0x303 (TLS 1.2)
      Random:
        gmt_unix_time=0xD3B13943
        random_bytes (len=28): 24530A97534115EA57B17C77ABAC33F909733FDBAA73C61AD594C525
      session_id (len=32): 397A7086775ACCDB82A75E2B19B7FE6459BF3088B322A7A5D8EAC17791A76D40
      cipher_suites (len=62)
        {0x13, 0x02} TLS_AES_256_GCM_SHA384
        {0x13, 0x03} TLS_CHACHA20_POLY1305_SHA256
        {0x13, 0x01} TLS_AES_128_GCM_SHA256
        {0xC0, 0x2C} TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
        {0xC0, 0x30} TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
        {0x00, 0x9F} TLS_DHE_RSA_WITH_AES_256_GCM_SHA384
        {0xCC, 0xA9} TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256
        {0xCC, 0xA8} TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256
        {0xCC, 0xAA} TLS_DHE_RSA_WITH_CHACHA20_POLY1305_SHA256
        {0xC0, 0x2B} TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
        {0xC0, 0x2F} TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
        {0x00, 0x9E} TLS_DHE_RSA_WITH_AES_128_GCM_SHA256
        {0xC0, 0x24} TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384
        {0xC0, 0x28} TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384
        {0x00, 0x6B} TLS_DHE_RSA_WITH_AES_256_CBC_SHA256
        {0xC0, 0x23} TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256
        {0xC0, 0x27} TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256
        {0x00, 0x67} TLS_DHE_RSA_WITH_AES_128_CBC_SHA256
        {0xC0, 0x0A} TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA
        {0xC0, 0x14} TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
        {0x00, 0x39} TLS_DHE_RSA_WITH_AES_256_CBC_SHA
        {0xC0, 0x09} TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA
        {0xC0, 0x13} TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
        {0x00, 0x33} TLS_DHE_RSA_WITH_AES_128_CBC_SHA
        {0x00, 0x9D} TLS_RSA_WITH_AES_256_GCM_SHA384
        {0x00, 0x9C} TLS_RSA_WITH_AES_128_GCM_SHA256
        {0x00, 0x3D} TLS_RSA_WITH_AES_256_CBC_SHA256
        {0x00, 0x3C} TLS_RSA_WITH_AES_128_CBC_SHA256
        {0x00, 0x35} TLS_RSA_WITH_AES_256_CBC_SHA
        {0x00, 0x2F} TLS_RSA_WITH_AES_128_CBC_SHA
        {0x00, 0xFF} TLS_EMPTY_RENEGOTIATION_INFO_SCSV
      compression_methods (len=1)
        No Compression (0x00)
      extensions, length = 188
        extension_type=server_name(0), length=14
          0000 - 00 0c 00 00 09 6c 6f 63-61 6c 68 6f 73 74      .....localhost
        extension_type=ec_point_formats(11), length=4
          uncompressed (0)
          ansiX962_compressed_prime (1)
          ansiX962_compressed_char2 (2)
        extension_type=supported_groups(10), length=6
          secp256r1 (P-256) (23)
          brainpoolP256r1 (26)
        extension_type=session_ticket(35), length=0
        extension_type=encrypt_then_mac(22), length=0
        extension_type=extended_master_secret(23), length=0
        extension_type=signature_algorithms(13), length=42
          ecdsa_secp256r1_sha256 (0x0403)
          ecdsa_secp384r1_sha384 (0x0503)
          ecdsa_secp521r1_sha512 (0x0603)
          ed25519 (0x0807)
          ed448 (0x0808)
          rsa_pss_pss_sha256 (0x0809)
          rsa_pss_pss_sha384 (0x080a)
          rsa_pss_pss_sha512 (0x080b)
          rsa_pss_rsae_sha256 (0x0804)
          rsa_pss_rsae_sha384 (0x0805)
          rsa_pss_rsae_sha512 (0x0806)
          rsa_pkcs1_sha256 (0x0401)
          rsa_pkcs1_sha384 (0x0501)
          rsa_pkcs1_sha512 (0x0601)
          ecdsa_sha224 (0x0303)
          rsa_pkcs1_sha224 (0x0301)
          dsa_sha224 (0x0302)
          dsa_sha256 (0x0402)
          dsa_sha384 (0x0502)
          dsa_sha512 (0x0602)
        extension_type=supported_versions(43), length=9
          TLS 1.3 (772)
          TLS 1.2 (771)
          TLS 1.1 (770)
          TLS 1.0 (769)
        extension_type=psk_key_exchange_modes(45), length=2
          psk_dhe_ke (1)
        extension_type=key_share(51), length=71
            NamedGroup: secp256r1 (P-256) (23)
            key_exchange:  (len=65): 04B9F00C469BB69090E6B7E8FC0EF518A600EEA9950825CFAFBC88DCF2B0EE218100ABD95B8C6CFAA29A7302332E42FCB07286C0653C823E5896981ED97C363D0D

Received Record
Header:
  Version = TLS 1.2 (0x303)
  Content Type = Alert (21)
  Length = 2
    Level=fatal(2), description=handshake failure(40)

40778BA11E7F0000:error:0A000410:SSL routines:ssl3_read_bytes:sslv3 alert handshake failure:ssl/record/rec_layer_s3.c:1605:SSL alert number 40
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 7 bytes and written 332 bytes
Verification: OK
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 0 (ok)
---

Reason is the openssl-3.2 server chooses TLS1.3 although there is no signature algorithm ecdsa_brainpoolP256r1_sha256 advertised by the openssl-3.0 client, so that is bound to fail. As a workaround either the openssl-3.0 client or the openssl-3.2 server could disable tls1_3 but that is not desirable for the RSA server.

[t8m]

If both versions support TLS-1.3 they must never choose older version. If they cannot agree on supported groups connection failure is inevitable.

[bernd-edlinger]

The problem is the client cannot know how the server is configured, and therefore the server should take the signature_algorithms extension into account, because the meaning of the ECDSA signature algorithm IDs are quite different for TLS1.3 and for TLS1.2: TLS1.3 implies a 1:1 connection between ECDSA signature algorithms and EC groups, but TLS1.2 allows the cross product of signature_algorithms extension and supported_groups extension. The server should know that, and choose TLS1.2 if it has the freedom to do so.

[kroeckx]

If both sides support TLS 1.3, that should be used. Otherwise you can force the use of older TLS versions.

[bernd-edlinger]

What you say means that, when the server does only support TLS1.2, then a man in the middle could simply remove any signature that he don't like from the signature_algorithms extension of the client hello, say everything with sha256, sha384 and sha512 in the name, and neither the server nor the client will notice anything from this attack?

Please say that there is some kind of a posteriori authentication of the unencrypted handshake messages exchanged between client and server, even if the TLS1.2 protocol is used.

[mattcaswell]

Please say that there is some kind of a posteriori authentication of the unencrypted handshake messages exchanged between client and server, even if the TLS1.2 protocol is used.

The entire handshake is authenticated in the Finished message.

[bernd-edlinger]

The entire handshake is authenticated in the Finished message.

Thanks for the confirmation. I want to use the received client hello's signature_algorithm extension in addition to the server's configured -sigalgs in is_tls13_capable to make the right decision which protocol to use.