Adding MSK IAM Auth to Kafka Connect image without workarounds

[mng1dev]

Hello,

I am trying to extend the cp-kafka-connect image so that it can support MSK authentication using IAM.

The first thing I did, was copying the jar in /usr/share/java/kafka, but this caused the following error:

Exception in thread "main" org.apache.kafka.common.config.ConfigException: Invalid value software.amazon.msk.auth.iam.IAMClientCallbackHandler for configuration sasl.client.callback.handler.class: Class software.amazon.msk.auth.iam.IAMClientCallbackHandler could not be found.

I copied the jar also in /usr/share/java/cp-base-new/, but the error persists.

Following some scattered clues, I found out that due to an issue in cub kafka-ready, the jar can be copied anywhere, but the ensure script inside the image entrypoint will prevent the image from starting. This is only true when the security.protocol property is not set to PLAINTEXT.

As a workaround I modified the run entrypoint by removing the call to ensure:

. /etc/confluent/docker/bash-config

. /etc/confluent/docker/mesos-setup.sh
. /etc/confluent/docker/apply-mesos-overrides

echo "===> User"
id

echo "===> Configuring ..."
/etc/confluent/docker/configure

# echo "===> Running preflight checks ... "
# /etc/confluent/docker/ensure

echo "===> Skipping preflight checks ... "

echo "===> Launching ... "
exec /etc/confluent/docker/launch

And now the docker image manages to connect to our MSK cluster without any problem.

Is there any way to avoid commenting that line and have the image working as expected?

[Devarsh23]

Hey @mng1dev , Were you able to find a concrete solution for this? We are facing the same issue.

[AnhHC]

Hey, I found the root cause and the solution. This is not about the ensure script, this is about the CLASSPATH problem when calling "/etc/confluent/docker/launch" script, the script will collect the necessary libraries in the CLASSPATH that you configured on the docker-compose script or the K8S yaml deployment, you will see the CLASSPATH environment, that's where you need to configure to put the jar files onto kafka-connect starting sequence.

Assuming the aws msk library location on the container which you already mounted is: /usr/share/java/schema-registry/aws-msk-iam-auth-xxx.jar so the corrected configuration in the CLASSPATH env (on K8s yaml file) is:

- name: CLASSPATH
          value: /usr/share/java/monitoring-interceptors/monitoring-interceptors-7.6.1.jar:/usr/share/java/schema-registry/aws-msk-iam-auth-xxx.jar

OR if you want to grab all jar files onto the CLASSPATH - specify the *star ()** in stead of the specific jar filename, just like this:

- name: CLASSPATH
          value: /usr/share/java/monitoring-interceptors/monitoring-interceptors-7.6.1.jar:/usr/share/java/schema-registry/*

And your kafka connect should work.

Hope this will help someone facing this problem. Cheers !