KMinion unable to open connection to broker : connection refused

[tanguynicolas]

Hello,

I have a very simple Kafka configuration, but KMinion container fail to connect to it anytimes.

This command works: kafka-topics.sh --version --bootstrap-server 127.0.0.1:9094

There is my Kafka configuration:

      KAFKA_ENABLE_KRAFT: yes                   # Default
      KAFKA_CFG_AUTO_CREATE_TOPICS_ENABLE: true # Default
      ALLOW_PLAINTEXT_LISTENER: yes             # Dev only (no SASL_SSL)
      KAFKA_CFG_LISTENERS: INSIDE://kafka:9092,OUTSIDE://kafka:9094,CONTROLLER://:9093
      KAFKA_CFG_ADVERTISED_LISTENERS: INSIDE://kafka:9092,OUTSIDE://localhost:9094
      KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP: INSIDE:PLAINTEXT,OUTSIDE:PLAINTEXT,CONTROLLER:PLAINTEXT
      KAFKA_CFG_INTER_BROKER_LISTENER_NAME: INSIDE
      KAFKA_CFG_CONTROLLER_LISTENER_NAMES: CONTROLLER
      KAFKA_OPTS: -javaagent:/bitnami/kafka/jmx_prometheus_javaagent.jar=12345:/bitnami/kafka/config/config.yaml

When I run docker run --rm -p 8080:8080 --env KAFKA_BROKERS=kafka:9092 redpandadata/kminion:v2.2.3 I get:

{"level":"info","ts":"2023-06-05T21:03:13.243Z","logger":"main","msg":"connecting to Kafka seed brokers, trying to fetch cluster metadata","seed_brokers":"kafka:9092"}
{"level":"warn","ts":"2023-06-05T21:03:13.244Z","logger":"main.kgo_client","msg":"unable to open connection to broker","addr":"kafka:9092","broker":"seed 0","err":"dial tcp 172.20.0.3:9092: connect: connection refused"}
{"level":"fatal","ts":"2023-06-05T21:03:13.244Z","logger":"main","msg":"failed to setup minion service","error":"failed to create kafka client: failed to test connectivity to Kafka cluster failed to request metadata: unable to dial: dial tcp 172.20.0.3:9092: connect: connection refused"}

I don't understand why :/ Could you help me?

Thanks for your sources, TN

[weeco]

It looks like you are connecting with KMinion via Docker where as you connect with the shell script from localhost. You must understand the concept of advertised listeners in Kafka, this is not specific to KMinion. Basically the reported advertised listener address must be reachable from the container where kminion is running and this doesn't seem to be the case here.

[tanguynicolas]

I don't know what happened, but when I tried again today, without changing anything in my configuration, it worked. That reassures me too, I had understood the listeners in Kafka. ^^ Sorry for the inconvenience

[tanguynicolas]

Ok I find the real solution.

Do not put hostname (kafka) below when using multi networks, cause Kafka will resolve it with IP of its own interface in FIRST network (by alphabetic order)

# DON'T USE
KAFKA_CFG_LISTENERS: INSIDE://kafka:9092,OUTSIDE://kafka:9094,CONTROLLER://:9093

# USE
KAFKA_CFG_LISTENERS: INSIDE://:9092,OUTSIDE://:9094,CONTROLLER://:9093

So the problem isn't Kminion problem.