randombit / botan

Cryptography Toolkit
https://botan.randombit.net
BSD 2-Clause "Simplified" License
2.59k stars 570 forks source link

Misusing `Credentials_Manager::cert_chain()` might lead to selection of incompatible ciphersuite #3256

Open marty1885 opened 1 year ago

marty1885 commented 1 year ago

Hi,

Sorry I have to ask here. I've searched and can't figure this out. I'm adding Botan as an alternative TLS provider to a networking library I maintain. I got both Botan's TLS server and client to work separately (I confirm this by pointing the client to Nginx and server can be accessed from ncat). However I can't get Botan's TLS client and server to talk to each other. Here's some relevant logs

Server:

20230207 08:26:34.999248 UTC 169303 TRACE [tls_emit_data] tls_emit_data: sending 9 bytes - BotanTLSConnectionImpl.cc:140
20230207 08:26:34.999738 UTC 169303 TRACE [onRecvMessage] Low level connection received 7 bytes. - BotanTLSConnectionImpl.cc:52
20230207 08:26:34.999767 UTC 169303 TRACE [tls_alert] tls_alert: illegal_parameter - BotanTLSConnectionImpl.cc:175
20230207 08:26:34.999809 UTC 169303 TRACE [handleClose] connection closed, fd=17 - TcpConnectionImpl.cc:307
20230207 08:26:34.999845 UTC 169303 TRACE [onConnection] Low level connection closed. - BotanTLSConnectionImpl.cc:44

Client

20230207 08:26:34.992580 UTC 169444 TRACE [tls_emit_data] tls_emit_data: sending 7 bytes - BotanTLSConnectionImpl.cc:140
20230207 08:26:34.992981 UTC 169444 ERROR Unexpected TLS Exception: Certificate key type did not match ciphersuite - BotanTLSConnectionImpl.cc:62
20230207 08:26:34.998957 UTC 169444 TRACE [onRecvMessage] Low level connection received 2127 bytes. - BotanTLSConnectionImpl.cc:52
20230207 08:26:34.999165 UTC 169444 ERROR Unexpected TLS Exception: Received handshake data after connection closure - BotanTLSConnectionImpl.cc:62

These lead me to believe this is a certificate issue. But I checked I'm using x509 v3 with X509v3 Extended Key Usage set to TLS Web Server Authentication, TLS Web Client Authentication via the following command

openssl req -new -newkey rsa:2048 -nodes -keyout tls.key -out tls.csr --subj '/C=TW/ST=Taipei/L=Taipei/O=Martin/OU=Martin/CN=Martin'
openssl req -x509 -days 365 -in tls.csr -signkey tls.key -out tls.crt -extensions v3_req -extfile <(echo -e "subjectAltName=DNS:localhost\nkeyUsage=digitalSignature,keyEncipherment\nextendedKeyUsage=serverAuth")

And I've confirmed it is V3 and with the correct key usage.

openssl x509 -in tls.crt -text
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            0e:a8:81:fd:49:7b:fb:58:4e:c7:c8:5c:f1:0c:4a:d2:e3:97:f8:d7
        Signature Algorithm: sha256WithRSAEncryption
        Issuer: C = TW, ST = Taipei, L = Taipei, O = Martin, OU = Martin, CN = Martin
        Validity
            Not Before: Feb  7 08:26:05 2023 GMT
            Not After : Feb  4 08:26:05 2033 GMT
        Subject: C = TW, ST = Taipei, L = Taipei, O = Martin, OU = Martin, CN = Martin
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (2048 bit)
                Modulus:
                    00:df:f8:a9:46:35:87:93:4b:4e:b8:be:4a:a0:5f:
                    f6:10:44:f5:84:15:08:d7:60:54:5c:7a:89:c9:19:
                    b9:33:bc:70:45:92:28:30:0f:91:24:b9:36:f0:f8:
                    4f:e9:e9:ef:d0:fc:5f:2c:25:7f:b4:d1:d2:f6:52:
                    3a:57:c9:38:cc:ce:ec:6f:a2:ff:86:07:e1:1b:2e:
                    5c:02:01:4c:c0:1e:55:cb:fb:ff:e5:51:4f:06:39:
                    bf:ff:a9:45:88:1a:07:69:98:30:da:6f:48:2e:93:
                    b1:41:57:f3:a4:d8:52:cd:98:7a:a8:e6:de:1d:9a:
                    82:e4:b4:50:14:fa:59:76:a8:b8:9c:02:24:55:27:
                    fd:66:ed:10:80:d2:fd:e8:b1:0b:27:6f:e4:05:dc:
                    28:0d:38:13:58:8b:c2:0d:6c:7c:80:da:b1:35:72:
                    ef:24:24:d3:d5:1b:fa:bb:83:45:45:1e:fe:a6:e2:
                    e5:de:47:f8:c1:dc:b0:7a:0b:c7:9c:cd:86:ce:88:
                    31:30:ce:d5:47:8c:85:74:78:e1:3d:26:02:16:41:
                    ad:42:af:9d:b4:c0:65:7a:f1:a7:a9:8a:91:ed:01:
                    ba:a4:e7:d4:4c:27:43:e7:7f:3a:ac:1e:3a:98:a1:
                    dd:61:3f:aa:07:c9:21:18:fa:a3:da:d8:3f:fe:4e:
                    ac:5f
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Subject Alternative Name: 
                DNS:localhost
            X509v3 Key Usage: 
                Digital Signature, Key Encipherment
            X509v3 Extended Key Usage: 
                TLS Web Server Authentication, TLS Web Client Authentication
            X509v3 Subject Key Identifier: 
                BC:28:E6:53:BA:9A:40:AD:51:00:36:D2:5E:D0:BD:08:51:F7:95:4A
    Signature Algorithm: sha256WithRSAEncryption
    Signature Value:
        da:63:a4:88:2d:03:9c:a8:4b:fb:1c:82:e8:f5:94:e6:de:e7:
        c5:89:b8:58:35:91:73:74:80:7f:a0:ce:10:9b:8f:bd:56:a1:
        5d:74:e0:c8:7c:cf:a4:c8:22:ad:9c:83:50:88:e0:06:0f:84:
        39:bf:9d:d6:9d:d6:7c:27:29:6e:08:2a:4b:11:05:cf:cf:5f:
        0c:87:fa:fa:c6:39:48:50:f5:73:58:fa:d0:73:3b:4e:f2:f1:
        f6:46:17:30:c9:8a:51:72:ef:38:3f:1c:d0:41:d8:24:82:1d:
        1c:cf:f7:14:2f:53:74:72:58:06:af:b5:e5:f4:5b:cf:cd:69:
        bd:8e:69:14:c1:72:29:65:7f:45:18:c6:3e:2c:f8:a4:c9:79:
        fb:3d:1b:60:ea:cd:86:8a:f8:13:28:a2:f1:74:b9:a2:ae:0c:
        69:df:4d:b9:af:47:a6:34:77:1b:03:51:ee:0e:47:75:b8:7d:
        73:df:a8:9a:68:38:a6:d3:ff:65:99:c9:07:0a:b5:e8:2d:b8:
        eb:d5:a6:e3:84:a0:bd:41:43:13:6d:82:82:77:93:de:7c:6d:
        bb:85:c6:a1:35:8f:3d:52:b0:9e:14:13:ca:2c:9c:f2:af:a1:
        81:43:67:72:89:8b:50:e8:03:b9:07:32:d5:7b:b5:e8:2e:d1:
        fc:f8:b8:5e
-----BEGIN CERTIFICATE-----
MIIDtTCCAp2gAwIBAgIUDqiB/Ul7+1hOx8hc8QxK0uOX+NcwDQYJKoZIhvcNAQEL
BQAwYjELMAkGA1UEBhMCVFcxDzANBgNVBAgMBlRhaXBlaTEPMA0GA1UEBwwGVGFp
cGVpMQ8wDQYDVQQKDAZNYXJ0aW4xDzANBgNVBAsMBk1hcnRpbjEPMA0GA1UEAwwG
TWFydGluMB4XDTIzMDIwNzA4MjYwNVoXDTMzMDIwNDA4MjYwNVowYjELMAkGA1UE
BhMCVFcxDzANBgNVBAgMBlRhaXBlaTEPMA0GA1UEBwwGVGFpcGVpMQ8wDQYDVQQK
DAZNYXJ0aW4xDzANBgNVBAsMBk1hcnRpbjEPMA0GA1UEAwwGTWFydGluMIIBIjAN
BgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA3/ipRjWHk0tOuL5KoF/2EET1hBUI
12BUXHqJyRm5M7xwRZIoMA+RJLk28PhP6env0PxfLCV/tNHS9lI6V8k4zM7sb6L/
hgfhGy5cAgFMwB5Vy/v/5VFPBjm//6lFiBoHaZgw2m9ILpOxQVfzpNhSzZh6qObe
HZqC5LRQFPpZdqi4nAIkVSf9Zu0QgNL96LELJ2/kBdwoDTgTWIvCDWx8gNqxNXLv
JCTT1Rv6u4NFRR7+puLl3kf4wdywegvHnM2GzogxMM7VR4yFdHjhPSYCFkGtQq+d
tMBlevGnqYqR7QG6pOfUTCdD5386rB46mKHdYT+qB8khGPqj2tg//k6sXwIDAQAB
o2MwYTAUBgNVHREEDTALgglsb2NhbGhvc3QwCwYDVR0PBAQDAgWgMB0GA1UdJQQW
MBQGCCsGAQUFBwMBBggrBgEFBQcDAjAdBgNVHQ4EFgQUvCjmU7qaQK1RADbSXtC9
CFH3lUowDQYJKoZIhvcNAQELBQADggEBANpjpIgtA5yoS/scguj1lObe58WJuFg1
kXN0gH+gzhCbj71WoV104Mh8z6TIIq2cg1CI4AYPhDm/ndad1nwnKW4IKksRBc/P
XwyH+vrGOUhQ9XNY+tBzO07y8fZGFzDJilFy7zg/HNBB2CSCHRzP9xQvU3RyWAav
teX0W8/Nab2OaRTBcillf0UYxj4s+KTJefs9G2DqzYaK+BMoovF0uaKuDGnfTbmv
R6Y0dxsDUe4OR3W4fXPfqJpoOKbT/2WZyQcKtegtuOvVpuOEoL1BQxNtgoJ3k958
bbuFxqE1jz1SsJ4UE8osnPKvoYFDZ3KJi1DoA7kHMtV7tegu0fz4uF4=
-----END CERTIFICATE-----

How do I generate a cert that Botan will accept?

marty1885 commented 1 year ago

NVM, I found there's the botan tls_server command. And that with my certs work. My TLS server has some issues.

reneme commented 1 year ago

Nevertheless: Judging from your log transcripts, I don't think your certificate is the problem. The client complains that the server selected a TLS cipher suite that does not allow RSA as signature algorithm. Maybe you excluded all such cipher suites by accident? E.g. via configuring Policy::allowed_signature_methods() or ::allowed_signature_schemes() or ::allowed_signature_hashes().

marty1885 commented 1 year ago

No, I'm using Botan::TLS::Default_Policy. I noticed the [tls_alert] tls_alert: illegal_parameter also shows up when I try to connect using openssl s_client. Does the issue sound familiar to you by chance?

I have ASan running. So can't be a buffer overflow somewhere messing this up.

reneme commented 1 year ago

No, it doesn't sound familiar but might be worth looking into. Are you using a stable version of Botan or latest master?

marty1885 commented 1 year ago

I'm using 2.19.3

reneme commented 1 year ago

Mhh, and I guess you just adapted an example to work with your network stack, right? I'd like to try and reproduce this. Or is this public code that you can share somewhere?

marty1885 commented 1 year ago

Yeah, I adapted the example on that page. Sorry for the late reply and thanks for your help, I was wiresharking too see if I can resolve it on my own.

The code is available here https://github.com/an-tao/trantor/pull/238. Code is a bit messy now as I haven't get to clean it yet. All Botan related code lives in trantor/net/inner/BotanTLSConnectionImpl.cc and the respective header. To replicate it run

Build:

git clone https://github.com/an-tao/trantor
git checkout tcpconnectionimpl-rewrite
mkdir build
cmake .. -DBUILD_TESTING=ON -DTRANTOR_PREFERED_TLS_PROVIDER="botan" -DCMAKE_BUILD_TYPE=Debug
make -j

Generate certificate (name should be server.cert and server.key)

openssl req -new -newkey rsa:2048 -nodes -keyout server.key -out server.csr --subj '/C=TW/ST=Taipei/L=Taipei/O=Martin/OU=Martin/CN=Martin'
openssl req -x509 -days 365 -in server.csr -signkey server.key -out server.cert -extensions v3_req -extfile <(echo -e "subjectAltName=DNS:localhost\nkeyUsage=digitalSignature,keyEncipherment\nextendedKeyUsage=serverAuth")

And run

# The TLS server that's causing issue. It'll print logs to console if CMake is in debug mode
./trantor/tests/ssl_server_test

# In another terminal
openssl s_client -connect localhost:8888 -tls1_2
reneme commented 1 year ago

I'm able to reproduce this. The server indeed selects the wrong ciphersuite for the certificate. My openssl s_client offers a long list of suites (including ones that would support RSA certificates for signing). But the Botan server implementation selected cc a9 - ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256 which expects an ECDSA certificate. So openssl s_client is right to complain.

Looking into it.

reneme commented 1 year ago

Found it! You tripped over a footgun in Botan's API. After applying this patch (and commenting out some faulty code in your override of tls_session_established) I managed to successfully establish a connection with openssl s_client and receive "Hello world". 😃

The issue is that Botan::TLS::Credentials_Manager::cert_chain() has a footgun. The TLS implementation tells you which cert_key_types it would like to receive. And it asks you three times (once for each RSA, ECDSA and DSA). Unfortunately, things get messed up if you're "lying" about the requested key type by just returning your certificate. As a result, the TLS implementation wrongly selected an ECDSA ciphersuite (believing you provided an ECDSA certificate) and OpenSSL (as well as the Botan-Client 🤡) rightfully complained about that.

@randombit Should we give this part of the Credentials_Manager some TLC before Botan 3.0 is coming out? Frankly, I was wondering about its design anyway. Particularly the ::cert_chain_single_type() indirection.

randombit commented 1 year ago

@reneme Sounds good to me. I'll take a look if I have time but this is iffy for me, at least within the next week.

marty1885 commented 1 year ago

@reneme Thanks! That would have took me ages to figure out. You are a savior.

Not sure if it'll be helpful. IMO it'll be good to update the document and the repo's example code to include the handling of this behavior. So others will see how to correctly handle it upon first look at the doc. I can make the changes if this sounds like a good idea.