Issue with Kafka Pod Communication Across Clusters in Bitnami

[akshaysolank]

Description: I am facing an issue with the deployment of a Kafka pod in Cluster A and its communication with microservices in Cluster B using a LoadBalancer (LB) defined for different brokers. Here are the details of the problem:

Cluster Configuration:

Cluster A: Contains the Kafka pod. Cluster B: Contains various microservices that need to communicate with the Kafka pod in Cluster A. Issue Description:

In Cluster B, we have defined the same Kafka LB for different brokers, and it works perfectly fine when microservices within Cluster A communicate with the Kafka pod. Problematic Behavior:

The issue arises when microservices in Cluster B try to communicate with the Kafka pod in Cluster A. We observe the following warning message in the Kafka logs:

2023-10-03 05:24:42.916 WARN 9 --- [| adminclient-1] org.apache.kafka.clients.NetworkClient : [AdminClient clientId=adminclient-1] Error connecting to node kafka-2.kafka-headless.kafka-pod.svc.cluster.local:9092 (id: 2 rack: null)

java.io.IOException: Can't resolve address: kafka-2.kafka-headless.kafka-pod.svc.cluster.local:9092
        at org.apache.kafka.common.network.Selector.doConnect(Selector.java:235) ~[classes!/:0.0.1-SNAPSHOT]
        at org.apache.kafka.common.network.Selector.connect(Selector.java:214) ~[classes!/:0.0.1-SNAPSHOT]
        at org.apache.kafka.clients.NetworkClient.initiateConnect(NetworkClient.java:864) [classes!/:0.0.1-SNAPSHOT]
        at org.apache.kafka.clients.NetworkClient.ready(NetworkClient.java:265) [classes!/:0.0.1-SNAPSHOT]
        at org.apache.kafka.clients.admin.KafkaAdminClient$AdminClientRunnable.sendEligibleCalls(KafkaAdminClient.java:889) [classes!/:0.0.1-SNAPSHOT]
        at org.apache.kafka.clients.admin.KafkaAdminClient$AdminClientRunnable.run(KafkaAdminClient.java:1104) [classes!/:0.0.1-SNAPSHOT]
        at java.lang.Thread.run(Thread.java:748) [na:1.8.0_191]
Caused by: java.nio.channels.UnresolvedAddressException: null
        at sun.nio.ch.Net.checkAddress(Net.java:101) ~[na:1.8.0_191]
        at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:622) ~[na:1.8.0_191]
        at org.apache.kafka.common.network.Selector.doConnect(Selector.java:233) ~[classes!/:0.0.1-SNAPSHOT]
        ... 6 common frames omitted

Problem Details:

The error message indicates that the Kafka pod in Cluster A is unable to resolve the address kafka-2.kafka-headless.kafka-pod.svc.cluster.local:9092 when attempting to connect from Cluster B. This issue is causing communication problems between the microservices in Cluster B and the Kafka pod in Cluster A. Expected Outcome:

We expect the Kafka pod in Cluster A to successfully resolve the address and establish communication with microservices in Cluster B via the defined LoadBalancer.

[akshaysolank]

Hello @migruiz4 could you please assist me on it

[OneCricketeer]

Bitnami has different charts. Are you sure this is the correct repo for you to be posting in?

Otherwise, please note #638

[akshaysolank]

I apologize for my mistake, @OneCricketeer. It appears that I posted the previous message on the wrong chat or thread.

[OneCricketeer]

Feel free to close this issue