Unable to connect from kafka client to the kafka server launched using helm chart in aws eks.

LingamEswar commented 1 day ago

Name and Version

bitnami/kafka version 28.0.0

What architecture are you using?

amd64

What steps will reproduce the bug?

Just connect to the existing eks cluster and created a namespace kafka and ran below command to install kafka using kafka helm chart version 28.0.0

The kafka is comming up and loadbalancer ip is able to access from outside. But when trying to connect to the kafka server using kafkaclient or using any other kafka ui docker images. Getting below issue .

When trying to connect for client , using command

docker run -p 8080:8080 -e KAFKA_CLUSTERS_0_NAME=kafka-cluster -e KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS=a2bf0471e16dc419ca6a6b8952a7d674-653566081.us-east-1.elb.amazonaws.com:9094 -e KAFKA_CLUSTERS_0_PROPERTIES_SECURITY_PROTOCOL=SASL_PLAINTEXT -e KAFKA_CLUSTERS_0_PROPERTIES_SASL_MECHANISM=SCRAM-SHA-256 -e KAFKA_CLUSTERS_0_PROPERTIES_SASL_JAAS_CONFIG="org.apache.kafka.common.security.scram.ScramLoginModule required username="mytestuser" password="J8gQYa7ersha";" provectuslabs/kafka-ui:latest

getting error like

[AdminClient clientId=kafka-ui-admin-1728998639-1] Connection to node -1 (a2bf0471e16dc419ca6a6b8952a7d674-653566081.us-east-1.elb.amazonaws.com/3.228.106.30:9094) terminated during authentication. This may happen due to any of the following reasons: (1) Authentication failed due to invalid credentials with brokers older than 1.0.0, (2) Firewall blocking Kafka TLS traffic (eg it may only allow HTTPS traffic), (3) Transient network issue

jaas config:

security.protocol=SASL_PLAINTEXT
sasl.mechanism=SCRAM-SHA-256
sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required \
    username="linge" \
    password="$(kubectl get secret kafkaa-user-passwords --namespace kafka -o jsonpath='{.data.client-passwords}' | base64 -d | cut -d , -f 1)";

even tried with below commands

kubectl run kafkaa-client --restart='Never' --image docker.io/bitnami/kafka:3.7.0-debian-12-r0 --namespace kafka --command -- sleep infinity
    kubectl cp --namespace kafka /path/to/client.properties kafkaa-client:/tmp/client.properties
    kubectl exec --tty -i kafkaa-client --namespace kafka -- bash

PRODUCER:

    kafka-console-producer.sh \
        --producer.config /tmp/client.properties \
        --broker-list kafkaa-controller-0.kafkaa-controller-headless.kafka.svc.cluster.local:9092 \
        --topic test

CONSUMER:

    kafka-console-consumer.sh \
        --consumer.config /tmp/client.properties \
        --bootstrap-server kafkaa.kafka.svc.cluster.local:9092 \
        --topic test \
        --from-beginning

Things are getting timed out.

Are you using any custom parameters or values?

Helm command:

helm install kafkaa bitnami/kafka -n kafka --version 28.0.0 --set externalAccess.enabled=true \
  --set externalAccess.service.broker.type=LoadBalancer \
  --set externalAccess.service.controller.type=LoadBalancer \
  --set externalAccess.service.broker.ports.external=9094 \
  --set externalAccess.service.controller.containerPorts.external=9094 \
  --set externalAccess.autoDiscovery.enabled=true \
  --set controller.replicaCount=1 \
  --set serviceAccount.create=true \
  --set rbac.create=true \
  --set controller.automountServiceAccountToken=true \
  --set broker.automountServiceAccountToken=true \
  --set sasl.client.users[0]=mytestuser\
  --set sasl.client.passwords[0]=J8gQYa7ersha \
  --set listeners.client.containerPort=9092 \
  --set listeners.client.name=CLIENT \
  --set listeners.controller.containerPort=9093 \
  --set listeners.controller.name=CONTROLLER \
  --set listeners.interbroker.containerPort=9094 \
  --set listeners.interbroker.name=INTERNAL \
  --set listeners.external.containerPort=9095 \
  --set listeners.external.name=EXTERNAL \
  --set superUsers=User:mytestuser

What is the expected behavior?

The client should connect to the server.

What do you see instead?

But facing timed out or

[SocketServer listenerType=BROKER, nodeId=0] Failed authentication with /172.21.39.46 (channelId=172.21.36.137:9095-172.21.39.46:54327-11) (Failing SASL authentication due to invalid receive size) (org.apache.kafka.common.network.Selector immediatley after server startup

[2024-10-15 11:57:36,954] INFO Kafka commitId: 2ae524ed625438c5 (org.apache.kafka.common.utils.AppInfoParser)
[2024-10-15 11:57:36,954] INFO Kafka startTimeMs: 1728993456954 (org.apache.kafka.common.utils.AppInfoParser)
[2024-10-15 11:57:36,955] INFO [KafkaRaftServer nodeId=0] Kafka Server started (kafka.server.KafkaRaftServer)
[2024-10-15 12:02:09,995] INFO [SocketServer listenerType=BROKER, nodeId=0] Failed authentication with /172.21.39.46 (channelId=172.21.36.137:9095-172.21.39.46:54327-11) (Failing SASL authentication due to invalid receive size) (org.apache.kafka.common.network.Selector)
[2024-10-15 12:04:13,966] INFO [SocketServer listenerType=BROKER, nodeId=0] Failed authentication with /172.21.39.46 (channelId=172.21.36.137:9095-172.21.39.46:63041-21) (Unexpected Kafka request of type METADATA during SASL handshake.) (org.apache.kafka.common.network.Selector)
[2024-10-15 12:04:14,362] INFO [SocketServer listenerType=BROKER, nodeId=0] Failed authentication with /172.21.39.46 (channelId=172.21.36.137:9095-172.21.39.46:38333-21) (Unexpected Kafka request of type METADATA during SASL handshake.) (org.apache.kafka.common.network.Selector)

LingamEswar commented 14 hours ago

@carrodher Any help here is much appreciated.

carrodher commented 11 hours ago

Hi, the issue may not be directly related to the Bitnami container image/Helm chart, but rather to how the application is being utilized, configured in your specific environment, or tied to a particular scenario that is not easy to reproduce on our side.

If you think that's not the case and want to contribute a solution, we welcome you to create a pull request. The Bitnami team is excited to review your submission and offer feedback. You can find the contributing guidelines here.

Your contribution will greatly benefit the community. Feel free to reach out if you have any questions or need assistance.

Suppose you have any questions about the application, customizing its content, or technology and infrastructure usage. In that case, we highly recommend that you refer to the forums and user guides provided by the project responsible for the application or technology.

With that said, we'll keep this ticket open until the stale bot automatically closes it, in case someone from the community contributes valuable insights.

LingamEswar commented 11 hours ago

@carrodher Just want to know, if is there any wrong with my configuration, I was using most of the configuration shown as per in values.yaml. But when the server is started facing an issues like.

[2024-10-16 06:16:57,317] INFO [SocketServer listenerType=BROKER, nodeId=0] Failed authentication with /172.21.39.46 (channelId=172.21.37.38:9095-172.21.39.46:13860-16) (Failing SASL authentication due to invalid receive size) (org.apache.kafka.common.network.Selector)

I was using all default protocols and mechanisms. So not sure why is that issue is coming. So just need your help on this.

bitnami / charts