SSL Kafka handshake failed over docker

[laurafbec]

Hi everyone, Based on the example https://github.com/confluentinc/cp-docker-images/tree/5.3.3-post/examples/kafka-cluster-ssl I've tried to develop a docker-compose file with zookeeper, a broker and the connect API with SSL enabled. The certificates have been generated by using the script included in the example and the content of the docker-compose file would be the next one:

`version: '3.6'

services: zookeeper: image: confluentinc/cp-zookeeper:7.0.1 hostname: zookeeper container_name: zookeeper ports:

• "2181:2181" environment: ZOOKEEPER_CLIENT_PORT: 2181 ZOOKEEPER_TICK_TIME: 2000 ZOOKEEPER_INIT_LIMIT: 5 ZOOKEEPER_SYNC_LIMIT: 2

  broker: image: confluentinc/cp-kafka:7.0.1 hostname: broker container_name: broker depends_on:

  • zookeeper ports:
  • "39093:39093"
  • "9093:9093" environment: KAFKA_BROKER_ID: 1 KAFKA_ZOOKEEPER_CONNECT: 'zookeeper:2181' KAFKA_ADVERTISED_LISTENERS: SSL://broker:39093,SSL_HOST://localhost:9093 KAFKA_SSL_KEYSTORE_FILENAME: kafka.broker.keystore.jks KAFKA_SSL_KEYSTORE_CREDENTIALS: broker_keystore_creds KAFKA_SSL_KEY_CREDENTIALS: broker_sslkey_creds KAFKA_SSL_TRUSTSTORE_FILENAME: kafka.broker.truststore.jks KAFKA_SSL_TRUSTSTORE_CREDENTIALS: broker_truststore_creds KAFKA_SSL_ENDPOINT_IDENTIFICATION_ALGORITHM: " " KAFKA_SSL_CLIENT_AUTH: required KAFKA_SECURITY_INTER_BROKER_PROTOCOL: SSL KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: SSL:SSL,SSL_HOST:SSL volumes:
  • $PWD/secrets2:/etc/kafka/secrets

  connect: image: confluentinc/cp-kafka-connect:7.0.1 hostname: connect container_name: connect depends_on:

  • zookeeper
  • broker ports:

  • "8083:8083" environment: CONNECT_BOOTSTRAP_SERVERS: 'broker:39093' CONNECT_REST_ADVERTISED_HOST_NAME: connect CONNECT_REST_PORT: 8083 CONNECT_GROUP_ID: compose-connect-group CONNECT_CONFIG_STORAGE_TOPIC: docker-connect-configs CONNECT_CONFIG_STORAGE_REPLICATION_FACTOR: 1 CONNECT_OFFSET_FLUSH_INTERVAL_MS: 10000 CONNECT_OFFSET_STORAGE_TOPIC: docker-connect-offsets CONNECT_OFFSET_STORAGE_REPLICATION_FACTOR: 1 CONNECT_STATUS_STORAGE_TOPIC: docker-connect-status CONNECT_STATUS_STORAGE_REPLICATION_FACTOR: 1 CONNECT_KEY_CONVERTER: org.apache.kafka.connect.json.JsonConverter CONNECT_VALUE_CONVERTER: org.apache.kafka.connect.json.JsonConverter CONNECT_INTERNAL_KEY_CONVERTER: "org.apache.kafka.connect.json.JsonConverter" CONNECT_INTERNAL_VALUE_CONVERTER: "org.apache.kafka.connect.json.JsonConverter" CONNECT_LOG4J_ROOT_LOGLEVEL: "INFO" CONNECT_LOG4J_LOGGERS: "org.apache.kafka.connect.runtime.rest=WARN,org.reflections=ERROR,com.mongodb.kafka=DEBUG" CONNECT_PLUGIN_PATH: /usr/share/confluent-hub-components CONNECT_ZOOKEEPER_CONNECT: 'zookeeper:2181' CLASSPATH: /usr/share/java/monitoring-interceptors/monitoring-interceptors-6.2.2.jar CONNECT_PRODUCER_INTERCEPTOR_CLASSES: "io.confluent.monitoring.clients.interceptor.MonitoringProducerInterceptor" CONNECT_CONSUMER_INTERCEPTOR_CLASSES: "io.confluent.monitoring.clients.interceptor.MonitoringConsumerInterceptor" CONNECT_SSL_KEYSTORE_FILENAME: kafka.connect.keystore.jks CONNECT_SSL_KEYSTORE_CREDENTIALS: connect_keystore_creds CONNECT_SSL_KEY_CREDENTIALS: connect_sslkey_creds CONNECT_SSL_TRUSTSTORE_FILENAME: kafka.connect.truststore.jks CONNECT_SSL_TRUSTSTORE_CREDENTIALS: connect_truststore_creds

    volumes:

  • $PWD/mongodb-kafka-connect:/usr/share/confluent-hub-components/kafka-connect-mongodb
  • $PWD/kafka/scripts:/scripts command:
  • bash
  • -c
  • | echo "Launching Kafka Connect worker" /etc/confluent/docker/run & # echo "Waiting for Kafka Connect to start listening on $$CONNECT_REST_ADVERTISED_HOST_NAME" while [ $$(curl -s -o /dev/null -w %{http_code} http://$$CONNECT_REST_ADVERTISED_HOST_NAME:$$CONNECT_REST_PORT/connectors) -ne 200 ]; do echo -e $$(date) "Kafka Connect listener HTTP state: "$$(curl -s -o /dev/null -w %{http_code} http://$$CONNECT_REST_ADVERTISED_HOST_NAME:$$CONNECT_REST_PORT/connectors)" (waiting for 200)" sleep 5 done nc -vz $$CONNECT_REST_ADVERTISED_HOST_NAME $$CONNECT_REST_PORT echo -e "\n--\n+> Creating Kafka Connect MongoDB sink" chmod 755 /scripts/sink-connect.sh echo -e "Permisos cambiados" /scripts/sink-connect.sh sleep infinity `

When running it I get the error from broker

[2022-01-10 11:08:03,163] INFO [SocketServer listenerType=ZK_BROKER, nodeId=1] Failed authentication with /172.19.0.4 (SSL handshake failed) (org.apache.kafka.common.network.Selector)

Can anyone help me? thanks in advance

[OneCricketeer]

The error is suggesting you didn't setup SSL for Zookeeper. And your environment variables for Zookeeper seem to confirm that

The examples have moved, by the way https://github.com/confluentinc/kafka-images/blob/master/examples/kafka-cluster-ssl/docker-compose.yml

[laurafbec]

Thanks @OneCricketeer !! I've actually solved it after posting the error. I had a mismatch between kafka listeners and the certs, and now I've TLSv1.3 enabled between kafka client and broker. I haven't found examples about enabling TLS for Zokeeper of Kafka connect. Do you know where can I find some of them? Thanks again!

[OneCricketeer]

haven't found examples about enabling TLS for Zookeeper or Kafka connect.

Is this page what you're looking for?

https://docs.confluent.io/platform/current/security/security_tutorial.html

Otherwise, Zookeeper has its own official documentation, and Connect is configured like any other broker client. Both utilize KAFKA_JAVA_OPTS environment variables for setting JAAS or other JVM System properties

Connect doesn't depend on Zookeeper

[laurafbec]

Thanks again @OneCricketeer!! I actually was looking for docker-compose examples with SSL enabled between zookeeper and Kafka. On that page is described SASL for Zookeeper, but, I think that SSL is not. But thanks anyway!! I'll check everything.