search

Blizzard / node-rdkafka

Node.js bindings for librdkafka
MIT License
2.1k stars 390 forks source link

"Broker did not provide a certificate" against aws MSK sasl/scram , due to not support of TLSv1.2 for handshake? #1005

Closed jun0tpyrc closed 1 year ago

jun0tpyrc commented 1 year ago

Environment Information

alpine nodejs14  dockerized
alpine nodejs16 dockerized
my localhost mac node16

Steps to Reproduce Create Kafka.HighLevelProducer with configs

node-rdkafka Configuration Settings simplified config

 configs["sasl.mechanism"] = "SCRAM-SHA-512"; // https://docs.aws.amazon.com/msk/latest/developerguide/msk-password.html#msk-password-limitations
 configs["security.protocol"] = "sasl_ssl";
& user & passwords etc

Additional context

jun0tpyrc commented 1 year ago

TLS handshake manually ok v1.2 to see cert , not for v1.1/v1.1 

openssl s_client -connect b-3.XXXXXXX.XXXX.c3.kafka.ap-southeast-1.amazonaws.com:9096

can see server cert

CONNECTED(00000005)
depth=2 C = US, O = Amazon, CN = Amazon Root CA 1
verify return:1
depth=1 C = US, O = Amazon, CN = Amazon RSA 2048 M02
verify return:1
depth=0 CN = *.xxxxx.xxxxxxx.c3.kafka.ap-southeast-1.amazonaws.com
verify return:1
---
Certificate chain
 0 s:CN = *.xxxxx.xxxx.c3.kafka.ap-southeast-1.amazonaws.com
   i:C = US, O = Amazon, CN = Amazon RSA 2048 M02
   a:PKEY: rsaEncryption, 2048 (bit); sigalg: RSA-SHA256
   v:NotBefore: Feb 23 00:00:00 2023 GMT; NotAfter: Mar 23 23:59:59 2024 GMT
 1 s:C = US, O = Amazon, CN = Amazon RSA 2048 M02
   i:C = US, O = Amazon, CN = Amazon Root CA 1
   a:PKEY: rsaEncryption, 2048 (bit); sigalg: RSA-SHA256
   v:NotBefore: Aug 23 22:25:30 2022 GMT; NotAfter: Aug 23 22:25:30 2030 GMT
 2 s:C = US, O = Amazon, CN = Amazon Root CA 1
   i:C = US, ST = Arizona, L = Scottsdale, O = "Starfield Technologies, Inc.", CN = Starfield Services Root Certificate Authority - G2
   a:PKEY: rsaEncryption, 2048 (bit); sigalg: RSA-SHA256
   v:NotBefore: May 25 12:00:00 2015 GMT; NotAfter: Dec 31 01:00:00 2037 GMT
 3 s:C = US, ST = Arizona, L = Scottsdale, O = "Starfield Technologies, Inc.", CN = Starfield Services Root Certificate Authority - G2
   i:C = US, O = "Starfield Technologies, Inc.", OU = Starfield Class 2 Certification Authority
   a:PKEY: rsaEncryption, 2048 (bit); sigalg: RSA-SHA256
   v:NotBefore: Sep  2 00:00:00 2009 GMT; NotAfter: Jun 28 17:39:16 2034 GMT
---
Server certificate
-----BEGIN CERTIFICATE-----
masked
-----END CERTIFICATE-----
subject=CN = *.xxxxx.xxxxx.c3.kafka.ap-southeast-1.amazonaws.com
issuer=C = US, O = Amazon, CN = Amazon RSA 2048 M02
---
No client certificate CA names sent
Peer signing digest: SHA256
Peer signature type: RSA-PSS
Server Temp Key: X25519, 253 bits
---
SSL handshake has read 5489 bytes and written 465 bytes
Verification: OK
---
New, TLSv1.2, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES256-GCM-SHA384
    Session-ID: XXXXX
    Session-ID-ctx:
    Master-Key: XXXXXXXX
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time: 1677213441
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
    Extended master secret: yes
jun0tpyrc commented 1 year ago 

> openssl s_client -connect b-3.xxxxxxx.xxxxx.c3.kafka.ap-southeast-1.amazonaws.com:9096 -tls1
CONNECTED(00000005)
0046CF1801000000:error:0A00010B:SSL routines:ssl3_get_record:wrong version number:ssl/record/ssl3_record.c:355:
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 5 bytes and written 179 bytes
Verification: OK
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1
    Cipher    : 0000
    Session-ID:
    Session-ID-ctx:
    Master-Key:
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time: 1677214152
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
    Extended master secret: no
---
> openssl s_client -connect b-3.XXXXXXXX.XXXXXX.c3.kafka.ap-southeast-1.amazonaws.com:9096 -tls1_1
CONNECTED(00000005)
0056761501000000:error:0A00010B:SSL routines:ssl3_get_record:wrong version number:ssl/record/ssl3_record.c:355:
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 5 bytes and written 179 bytes
Verification: OK
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.1
    Cipher    : 0000
    Session-ID:
    Session-ID-ctx:
    Master-Key:
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time: 1677214154
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
    Extended master secret: no
---
jun0tpyrc commented 1 year ago

link back to https://github.com/confluentinc/librdkafka/issues/2723 seem

jun0tpyrc commented 1 year ago
mogopz commented 1 year ago

@jun0tpyrc Did you ever get this working with Alpine? I'm going through the process of upgrading from Alpine 3.16 to 3.17 and librdkafka 1.7.0 to 2.0.2 and am seeing the same Broker did not provide a certificate error which must be related to the OpenSSL v3 upgrade. Building the images takes a long time so it's pretty painful to debug

jun0tpyrc commented 1 year ago

no, we giving up alpine switching back to debian based image to use precompiled package (tried various effort compiling own rdkafka with openssl etc, did not figure out it )

@jun0tpyrc Did you ever get this working with Alpine? I'm going through the process of upgrading from Alpine 3.16 to 3.17 and librdkafka 1.7.0 to 2.0.2 and am seeing the same Broker did not provide a certificate error which must be related to the OpenSSL v3 upgrade. Building the images takes a long time so it's pretty painful to debug