Unable to connect with Kafka Cluster while kafka-topics.sh can using security.protocol=SSL

[fr33m0nk]

Hi,

I was trying to use Kafka-proxy for AWS MSK. I was running Kafka-proxy from a EC2 deployed in the same VPC as that of AWS MSK. AWS MSK has Unauthenticated access enabled and following auth schemes disabled:

• TLS client authentication through AWS Certificate Manager (ACM)
• SASL/SCRAM authentication
• IAM role-based authentication

I can connect to AWS MSK via kafka-topics.sh using security.protocol=SSL from EC2. AWS MSK has Kafka brokers that can be connected only on port 9094. I have tried --tls-enable and --tls-insecure-skip-verify but it didn't work. It would be great if a little guidance is provided.

[fr33m0nk]

I managed to get around and solve this. For a AWS MSK configuration like below, a self signed certificate needs to be generated: AWS MSK has Unauthenticated access enabled and following auth schemes disabled:

• TLS client authentication through AWS Certificate Manager (ACM)
• SASL/SCRAM authentication
• IAM role-based authentication

Using the self signed certificate, Kafka-proxy started blazing :)

Huge shoutout to the contributors for the amazing work!!