TLS with PKCS12 keystore - getting client SSL authentication might be required

[Constantin07]

I'm trying to connect to AWS MSK cluster, configured with mTLS but getting this error:

$ kcat -b "$BROKERS" -F kcat.config -L
% Reading configuration from file kcat.config
%3|1656605146.834|FAIL|rdkafka#producer-1| [thrd:ssl://b-1.***.kafka.eu-west-1.amazo]: ssl://b-1.***.kafka.eu-west-1.amazonaws.com:9094/bootstrap: SSL handshake failed: error:141A10F4:SSL routines:ossl_statem_client_read_transition:unexpected message: client SSL authentication might be required (see ssl.key.location and ssl.certificate.location and consult the broker logs for more information) (after 16ms in state SSL_HANDSHAKE)
%3|1656605147.831|FAIL|rdkafka#producer-1| [thrd:ssl://b-1.***.kafka.eu-west-1.amazo]: ssl://b-1.***.kafka.eu-west-1.amazonaws.com:9094/bootstrap: SSL handshake failed: error:141A10F4:SSL routines:ossl_statem_client_read_transition:unexpected message: client SSL authentication might be required (see ssl.key.location and ssl.certificate.location and consult the broker logs for more information) (after 17ms in state SSL_HANDSHAKE, 1 identical error(s) suppressed)
% ERROR: Failed to acquire metadata: Local: Broker transport failure (Are the brokers reachable? Also try increasing the metadata timeout with -m <timeout>?)

Kcat config file:

$ cat kcat.config
security.protocol=ssl
ssl.endpoint.identification.algorithm=
ssl.keystore.location=/home/user/kafka.admin.keystore.p12
ssl.keystore.password=***************

With the same PKCS12 keystore and password in the kafka.admin.properties file:

$ cat kafka.admin.properties
security.protocol=SSL
ssl.enabled.protocols=TLSv1.2
ssl.truststore.type=PKCS12
ssl.truststore.location=/home/mskuser/kafka.admin.keystore.p12
ssl.truststore.password=***************
ssl.keystore.location=/home/mskuser/kafka.admin.keystore.p12
ssl.keystore.password=***************
ssl.key.password=***************
ssl.endpoint.identification.algorithm=

the Kafka tools work just fine:

$ kafka-topics.sh --bootstrap-server $BROKERS --command-config kafka.admin.properties --list
<list of topics here>

Alpine (Docker container): 3.15.4 OpenSSL version: OpenSSL 1.1.1p Kcat version:

kcat -V
kcat - Apache Kafka producer and consumer tool
https://github.com/edenhill/kcat
Copyright (c) 2014-2021, Magnus Edenhill
Version v3.15.0_alpha20210804-4411-g6ef8a4 (JSON, Transactions, IncrementalAssign, librdkafka 1.8.2 builtin.features=gzip,snappy,ssl,sasl,regex,lz4,sasl_gssapi,sasl_plain,sasl_scram,plugins,zstd,sasl_oauthbearer)

NB: In the PKCS12 keystore I've got:

• client public/private keypair
• client signed certificate
• Root & Intermediate certs used to sign client cert
• AWS Root/Intermediate CA chain (as trusted) used for verifying Broker certs

I'm wondering if kcat does support the PKCS12 formatted keystores ?

[britoaldan]

hi @Constantin07 Its not just with the PKCS12 format, even the jks or pem format certifcates result in the same. kafka-console-producers and consumers work fine. hi @edenhill, does kcat utility support openssl 1.1.1-any with librdkafka 1.8.2 which uses Tls1.3 protocol

[edenhill]

Yes, librdkafka 1.8.2 supports OpenSSL 1.0.2 and 1.1. librdkafka 2.0 also supports OpenSSL 3.0.

Enable security debugging to see which OpenSSL version is being used: kcat …. -d security

tis 14 feb. 2023 kl. 10:05 skrev Aldan Basil Brito @.***

:

hi @Constantin07 https://github.com/Constantin07 Its not just with the PKCS12 format, even the jks or pem format certifcates result in the same. kafka-console-producers and consumers work fine. hi @edenhill https://github.com/edenhill, does kcat utility support openssl 1.1.1-any with librdkafka 1.8.2 which uses Tls1.3 protocol

— Reply to this email directly, view it on GitHub https://github.com/edenhill/kcat/issues/384#issuecomment-1429367597, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAEAFPQPOQXTFIK4IVB2UY3WXNDHBANCNFSM52J436NQ . You are receiving this because you were mentioned.Message ID: @.***>