search

bitnami / containers

Bitnami container images
https://bitnami.com
Other
3.25k stars 4.75k forks source link

[Kafka] pods are exiting when deployed kafka with kraft mode using docker compose #67247

Closed achar007 closed 2 months ago

achar007 commented 3 months ago

Name and Version

bitnami/kafka:latest

What architecture are you using?

amd64

What steps will reproduce the bug?

Here's my docker-compose.yml file

version: '2'

services:
  kafka-combined:
    image: docker.io/bitnami/kafka:latest
    ports:
      - "9092:9092"
    environment:
      - KAFKA_CFG_NODE_ID=0
      - KAFKA_CFG_PROCESS_ROLES=controller,broker
      - KAFKA_CFG_LISTENERS=PLAINTEXT://:9092,CONTROLLER://:9093
      - KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT
      - KAFKA_CFG_CONTROLLER_QUORUM_VOTERS=0@kafka-0:9093,1@kafka-1:9093
      - KAFKA_CFG_CONTROLLER_LISTENER_NAMES=CONTROLLER
      - KAFKA_KRAFT_CLUSTER_ID=abcdefghijklmnopqrstuv
    volumes:
      - kafka_0_data:/bitnami/kafka
  kafka-controller:
    image: docker.io/bitnami/kafka:latest
    environment:
      - KAFKA_CFG_NODE_ID=1
      - KAFKA_CFG_PROCESS_ROLES=controller
      - KAFKA_CFG_LISTENERS=CONTROLLER://:9093
      - KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=CONTROLLER:PLAINTEXT
      - KAFKA_CFG_CONTROLLER_QUORUM_VOTERS=0@kafka-0:9093,1@kafka-1:9093
      - KAFKA_CFG_CONTROLLER_LISTENER_NAMES=CONTROLLER
      - KAFKA_KRAFT_CLUSTER_ID=abcdefghijklmnopqrstuv
    volumes:
      - kafka_1_data:/bitnami/kafka
  kafka-broker:
    image: docker.io/bitnami/kafka:latest
    environment:
      - KAFKA_CFG_NODE_ID=2
      - KAFKA_CFG_PROCESS_ROLES=broker
      - KAFKA_CFG_CONTROLLER_QUORUM_VOTERS=0@kafka-0:9093,1@kafka-1:9093
    volumes:
      - kafka_2_data:/bitnami/kafka

volumes:
  kafka_0_data:
    driver: local
  kafka_1_data:
    driver: local
  kafka_2_data:
    driver: local

$ docker compose up -d
WARN[0000] /root/docker-compose.yml: `version` is obsolete 
[+] Running 4/4
 ✔ kafka-controller Pulled                                                                                                                                                                           93.0s 
 ✔ kafka-broker Pulled                                                                                                                                                                               93.0s 
   ✔ 9bb44e3263fd Pull complete                                                                                                                                                                      87.8s 
 ✔ kafka-combined Pulled                                                                                                                                                                             93.0s 
[+] Running 7/7
 ✔ Network root_default               Created                                                                                                                                                         0.1s 
 ✔ Volume "root_kafka_2_data"         Created                                                                                                                                                         0.0s 
 ✔ Volume "root_kafka_0_data"         Created                                                                                                                                                         0.0s 
 ✔ Volume "root_kafka_1_data"         Created                                                                                                                                                         0.0s 
 ✔ Container root-kafka-controller-1  Started                                                                                                                                                         1.5s 
 ✔ Container root-kafka-broker-1      Started                                                                                                                                                         1.5s 
 ✔ Container root-kafka-combined-1    Started                                                                                                                                                         1.5s 
root@bharath-A320MA-M-2:~# docker ps
$ docker ps
CONTAINER ID   IMAGE                  COMMAND                  CREATED         STATUS         PORTS                                       NAMES
d12f210c5d4a   bitnami/kafka:latest   "/opt/bitnami/script…"   7 seconds ago   Up 5 seconds   0.0.0.0:9092->9092/tcp, :::9092->9092/tcp   root-kafka-combined-1
11ed926a2645   bitnami/kafka:latest   "/opt/bitnami/script…"   7 seconds ago   Up 5 seconds   9092/tcp                                    root-kafka-controller-1
85803ac161f4   bitnami/kafka:latest   "/opt/bitnami/script…"   7 seconds ago   Up 5 seconds   9092/tcp                                    root-kafka-broker-1

$ docker ps -a
CONTAINER ID   IMAGE                  COMMAND                  CREATED         STATUS                      PORTS      NAMES
d12f210c5d4a   bitnami/kafka:latest   "/opt/bitnami/script…"   4 minutes ago   Exited (1) 2 minutes ago               root-kafka-combined-1
11ed926a2645   bitnami/kafka:latest   "/opt/bitnami/script…"   4 minutes ago   Up 4 minutes                9092/tcp   root-kafka-controller-1
85803ac161f4   bitnami/kafka:latest   "/opt/bitnami/script…"   4 minutes ago   Exited (1) 3 minutes ago               root-kafka-broker-1
ab934a4c5039   2418e8f8ffa3           "/bin/bash"              4 months ago    Exited (130) 4 months ago              my-os-shell-container
$ docker logs d12f210c5d4a
kafka 15:53:51.46 INFO  ==> 
kafka 15:53:51.46 INFO  ==> Welcome to the Bitnami kafka container
kafka 15:53:51.46 INFO  ==> Subscribe to project updates by watching https://github.com/bitnami/containers
kafka 15:53:51.46 INFO  ==> Submit issues and feature requests at https://github.com/bitnami/containers/issues
kafka 15:53:51.47 INFO  ==> Upgrade to Tanzu Application Catalog for production environments to access custom-configured and pre-packaged software components. Gain enhanced features, including Software Bill of Materials (SBOM), CVE scan result reports, and VEX documents. To learn more, visit https://bitnami.com/enterprise
kafka 15:53:51.47 INFO  ==> 
kafka 15:53:51.47 INFO  ==> ** Starting Kafka setup **
kafka 15:53:53.59 WARN  ==> Kafka has been configured with a PLAINTEXT listener, this setting is not recommended for production environments.
kafka 15:53:53.62 INFO  ==> Initializing Kafka...
kafka 15:53:53.63 INFO  ==> No injected configuration files found, creating default config files
kafka 15:53:54.04 INFO  ==> Initializing KRaft storage metadata
kafka 15:53:54.05 INFO  ==> Formatting storage directories to add metadata...
metaPropertiesEnsemble=MetaPropertiesEnsemble(metadataLogDir=Optional.empty, dirs={/bitnami/kafka/data: EMPTY})
Formatting /bitnami/kafka/data with metadata.version 3.7-IV4.

kafka 15:53:57.22 INFO  ==> ** Kafka setup finished! **
kafka 15:53:57.24 INFO  ==> ** Starting Kafka **
[2024-05-25 15:53:59,006] INFO Registered kafka:type=kafka.Log4jController MBean (kafka.utils.Log4jControllerRegistration$)
[2024-05-25 15:53:59,686] INFO Setting -D jdk.tls.rejectClientInitiatedRenegotiation=true to disable client-initiated TLS renegotiation (org.apache.zookeeper.common.X509Util)
[2024-05-25 15:54:00,032] INFO Registered signal handlers for TERM, INT, HUP (org.apache.kafka.common.utils.LoggingSignalHandler)
[2024-05-25 15:54:00,039] INFO [ControllerServer id=0] Starting controller (kafka.server.ControllerServer)
[2024-05-25 15:54:01,132] INFO Updated connection-accept-rate max connection creation rate to 2147483647 (kafka.network.ConnectionQuotas)
[2024-05-25 15:54:01,204] INFO [SocketServer listenerType=CONTROLLER, nodeId=0] Created data-plane acceptor and processors for endpoint : ListenerName(CONTROLLER) (kafka.network.SocketServer)
[2024-05-25 15:54:01,209] INFO CONTROLLER: resolved wildcard host to d12f210c5d4a (org.apache.kafka.metadata.ListenerInfo)
[2024-05-25 15:54:01,224] INFO authorizerStart completed for endpoint CONTROLLER. Endpoint is now READY. (org.apache.kafka.server.network.EndpointReadyFutures)
[2024-05-25 15:54:01,225] INFO [SharedServer id=0] Starting SharedServer (kafka.server.SharedServer)
[2024-05-25 15:54:01,401] INFO [LogLoader partition=__cluster_metadata-0, dir=/bitnami/kafka/data] Loading producer state till offset 0 with message format version 2 (kafka.log.UnifiedLog$)
[2024-05-25 15:54:01,403] INFO [LogLoader partition=__cluster_metadata-0, dir=/bitnami/kafka/data] Reloading from producer snapshot and rebuilding producer state from offset 0 (kafka.log.UnifiedLog$)
[2024-05-25 15:54:01,403] INFO [LogLoader partition=__cluster_metadata-0, dir=/bitnami/kafka/data] Producer state recovery took 0ms for snapshot load and 0ms for segment recovery from offset 0 (kafka.log.UnifiedLog$)
[2024-05-25 15:54:01,462] INFO Initialized snapshots with IDs Set() from /bitnami/kafka/data/__cluster_metadata-0 (kafka.raft.KafkaMetadataLog$)
[2024-05-25 15:54:01,502] INFO [raft-expiration-reaper]: Starting (kafka.raft.TimingWheelExpirationService$ExpiredOperationReaper)
[2024-05-25 15:54:01,821] INFO [RaftManager id=0] Completed transition to Unattached(epoch=0, voters=[0, 1], electionTimeoutMs=1418) from null (org.apache.kafka.raft.QuorumState)
[2024-05-25 15:54:01,826] INFO [kafka-0-raft-outbound-request-thread]: Starting (org.apache.kafka.raft.KafkaNetworkChannel$SendThread)
[2024-05-25 15:54:01,829] INFO [kafka-0-raft-io-thread]: Starting (kafka.raft.KafkaRaftManager$RaftIoThread)
[2024-05-25 15:54:01,861] INFO [ControllerServer id=0] Waiting for controller quorum voters future (kafka.server.ControllerServer)
[2024-05-25 15:54:01,862] INFO [ControllerServer id=0] Finished waiting for controller quorum voters future (kafka.server.ControllerServer)
[2024-05-25 15:54:01,864] INFO [RaftManager id=0] Registered the listener org.apache.kafka.image.loader.MetadataLoader@1223443194 (org.apache.kafka.raft.KafkaRaftClient)
[2024-05-25 15:54:01,869] INFO [MetadataLoader id=0] initializeNewPublishers: the loader is still catching up because we still don't know the high water mark yet. (org.apache.kafka.image.loader.MetadataLoader)
[2024-05-25 15:54:01,943] INFO [QuorumController id=0] Creating new QuorumController with clusterId abcdefghijklmnopqrstuv. (org.apache.kafka.controller.QuorumController)
[2024-05-25 15:54:01,944] INFO [RaftManager id=0] Registered the listener org.apache.kafka.controller.QuorumController$QuorumMetaLogListener@869743379 (org.apache.kafka.raft.KafkaRaftClient)
[2024-05-25 15:54:01,957] INFO [controller-0-ThrottledChannelReaper-Fetch]: Starting (kafka.server.ClientQuotaManager$ThrottledChannelReaper)
[2024-05-25 15:54:01,960] INFO [controller-0-ThrottledChannelReaper-Produce]: Starting (kafka.server.ClientQuotaManager$ThrottledChannelReaper)
[2024-05-25 15:54:01,963] INFO [controller-0-ThrottledChannelReaper-Request]: Starting (kafka.server.ClientQuotaManager$ThrottledChannelReaper)
[2024-05-25 15:54:01,969] INFO [controller-0-ThrottledChannelReaper-ControllerMutation]: Starting (kafka.server.ClientQuotaManager$ThrottledChannelReaper)
[2024-05-25 15:54:01,970] INFO [MetadataLoader id=0] initializeNewPublishers: the loader is still catching up because we still don't know the high water mark yet. (org.apache.kafka.image.loader.MetadataLoader)
[2024-05-25 15:54:02,016] INFO [ExpirationReaper-0-AlterAcls]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2024-05-25 15:54:02,048] INFO [MetadataLoader id=0] initializeNewPublishers: the loader is still catching up because we still don't know the high water mark yet. (org.apache.kafka.image.loader.MetadataLoader)
[2024-05-25 15:54:02,049] INFO [ControllerServer id=0] Waiting for the controller metadata publishers to be installed (kafka.server.ControllerServer)
[2024-05-25 15:54:02,049] INFO [ControllerServer id=0] Finished waiting for the controller metadata publishers to be installed (kafka.server.ControllerServer)
[2024-05-25 15:54:02,050] INFO [SocketServer listenerType=CONTROLLER, nodeId=0] Enabling request processing. (kafka.network.SocketServer)
[2024-05-25 15:54:02,059] INFO Awaiting socket connections on 0.0.0.0:9093. (kafka.network.DataPlaneAcceptor)
[2024-05-25 15:54:02,111] INFO [controller-0-to-controller-registration-channel-manager]: Starting (kafka.server.NodeToControllerRequestThread)
[2024-05-25 15:54:02,111] INFO [ControllerServer id=0] Waiting for all of the authorizer futures to be completed (kafka.server.ControllerServer)
[2024-05-25 15:54:02,111] INFO [ControllerServer id=0] Finished waiting for all of the authorizer futures to be completed (kafka.server.ControllerServer)
[2024-05-25 15:54:02,111] INFO [ControllerServer id=0] Waiting for all of the SocketServer Acceptors to be started (kafka.server.ControllerServer)
[2024-05-25 15:54:02,111] INFO [ControllerServer id=0] Finished waiting for all of the SocketServer Acceptors to be started (kafka.server.ControllerServer)
[2024-05-25 15:54:02,114] INFO [BrokerServer id=0] Transition from SHUTDOWN to STARTING (kafka.server.BrokerServer)
[2024-05-25 15:54:02,115] INFO [ControllerRegistrationManager id=0 incarnation=4Qpjfb_rSQ69SeLT1U75RQ] initialized channel manager. (kafka.server.ControllerRegistrationManager)
[2024-05-25 15:54:02,116] INFO [BrokerServer id=0] Starting broker (kafka.server.BrokerServer)
[2024-05-25 15:54:02,117] INFO [ControllerRegistrationManager id=0 incarnation=4Qpjfb_rSQ69SeLT1U75RQ] maybeSendControllerRegistration: cannot register yet because the metadata version is still 3.0-IV1, which does not support KIP-919 controller registration. (kafka.server.ControllerRegistrationManager)
[2024-05-25 15:54:02,138] INFO [broker-0-ThrottledChannelReaper-Fetch]: Starting (kafka.server.ClientQuotaManager$ThrottledChannelReaper)
[2024-05-25 15:54:02,138] INFO [broker-0-ThrottledChannelReaper-Produce]: Starting (kafka.server.ClientQuotaManager$ThrottledChannelReaper)
[2024-05-25 15:54:02,139] INFO [broker-0-ThrottledChannelReaper-Request]: Starting (kafka.server.ClientQuotaManager$ThrottledChannelReaper)
[2024-05-25 15:54:02,143] INFO [broker-0-ThrottledChannelReaper-ControllerMutation]: Starting (kafka.server.ClientQuotaManager$ThrottledChannelReaper)
[2024-05-25 15:54:02,149] INFO [MetadataLoader id=0] initializeNewPublishers: the loader is still catching up because we still don't know the high water mark yet. (org.apache.kafka.image.loader.MetadataLoader)
[2024-05-25 15:54:02,204] INFO [BrokerServer id=0] Waiting for controller quorum voters future (kafka.server.BrokerServer)
[2024-05-25 15:54:02,205] INFO [BrokerServer id=0] Finished waiting for controller quorum voters future (kafka.server.BrokerServer)
[2024-05-25 15:54:02,220] INFO [broker-0-to-controller-forwarding-channel-manager]: Starting (kafka.server.NodeToControllerRequestThread)
[2024-05-25 15:54:02,251] INFO [MetadataLoader id=0] initializeNewPublishers: the loader is still catching up because we still don't know the high water mark yet. (org.apache.kafka.image.loader.MetadataLoader)
[2024-05-25 15:54:02,329] INFO Updated connection-accept-rate max connection creation rate to 2147483647 (kafka.network.ConnectionQuotas)
[2024-05-25 15:54:02,337] INFO [SocketServer listenerType=BROKER, nodeId=0] Created data-plane acceptor and processors for endpoint : ListenerName(PLAINTEXT) (kafka.network.SocketServer)
[2024-05-25 15:54:02,339] INFO PLAINTEXT: resolved wildcard host to d12f210c5d4a (org.apache.kafka.metadata.ListenerInfo)
[2024-05-25 15:54:02,351] INFO [broker-0-to-controller-alter-partition-channel-manager]: Starting (kafka.server.NodeToControllerRequestThread)
[2024-05-25 15:54:02,352] INFO [MetadataLoader id=0] initializeNewPublishers: the loader is still catching up because we still don't know the high water mark yet. (org.apache.kafka.image.loader.MetadataLoader)
[2024-05-25 15:54:02,365] INFO [broker-0-to-controller-directory-assignments-channel-manager]: Starting (kafka.server.NodeToControllerRequestThread)
[2024-05-25 15:54:02,395] INFO [ExpirationReaper-0-Produce]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2024-05-25 15:54:02,398] INFO [ExpirationReaper-0-Fetch]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2024-05-25 15:54:02,401] INFO [ExpirationReaper-0-DeleteRecords]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2024-05-25 15:54:02,404] INFO [ExpirationReaper-0-ElectLeader]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2024-05-25 15:54:02,406] INFO [ExpirationReaper-0-RemoteFetch]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2024-05-25 15:54:02,453] INFO [ExpirationReaper-0-Heartbeat]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2024-05-25 15:54:02,453] INFO [MetadataLoader id=0] initializeNewPublishers: the loader is still catching up because we still don't know the high water mark yet. (org.apache.kafka.image.loader.MetadataLoader)
[2024-05-25 15:54:02,455] INFO [ExpirationReaper-0-Rebalance]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2024-05-25 15:54:02,517] INFO Unable to read the broker epoch in /bitnami/kafka/data. (kafka.log.LogManager)
[2024-05-25 15:54:02,523] INFO [broker-0-to-controller-heartbeat-channel-manager]: Starting (kafka.server.NodeToControllerRequestThread)
[2024-05-25 15:54:02,531] INFO [BrokerLifecycleManager id=0] Incarnation oREZbJVtTPWGFm9ErvKdpQ of broker 0 in cluster abcdefghijklmnopqrstuv is now STARTING. (kafka.server.BrokerLifecycleManager)
[2024-05-25 15:54:02,549] INFO [ExpirationReaper-0-AlterAcls]: Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2024-05-25 15:54:02,555] INFO [MetadataLoader id=0] initializeNewPublishers: the loader is still catching up because we still don't know the high water mark yet. (org.apache.kafka.image.loader.MetadataLoader)
[2024-05-25 15:54:02,580] INFO [BrokerServer id=0] Waiting for the broker metadata publishers to be installed (kafka.server.BrokerServer)
[2024-05-25 15:54:02,580] INFO [MetadataLoader id=0] initializeNewPublishers: the loader is still catching up because we still don't know the high water mark yet. (org.apache.kafka.image.loader.MetadataLoader)
[2024-05-25 15:54:02,580] INFO [BrokerServer id=0] Finished waiting for the broker metadata publishers to be installed (kafka.server.BrokerServer)
[2024-05-25 15:54:02,581] INFO [BrokerServer id=0] Waiting for the controller to acknowledge that we are caught up (kafka.server.BrokerServer)
[2024-05-25 15:54:02,681] INFO [MetadataLoader id=0] initializeNewPublishers: the loader is still catching up because we still don't know the high water mark yet. (org.apache.kafka.image.loader.MetadataLoader)
[2024-05-25 15:54:02,782] INFO [MetadataLoader id=0] initializeNewPublishers: the loader is still catching up because we still don't know the high water mark yet. (org.apache.kafka.image.loader.MetadataLoader)
[2024-05-25 15:54:02,882] INFO [MetadataLoader id=0] initializeNewPublishers: the loader is still catching up because we still don't know the high water mark yet. (org.apache.kafka.image.loader.MetadataLoader)
[2024-05-25 15:54:02,961] INFO [RaftManager id=0] Completed transition to CandidateState(localId=0, epoch=1, retries=1, voteStates={0=GRANTED, 1=UNRECORDED}, highWatermark=Optional.empty, electionTimeoutMs=1783) from Unattached(epoch=0, voters=[0, 1], electionTimeoutMs=1418) (org.apache.kafka.raft.QuorumState)
[2024-05-25 15:54:02,965] INFO [QuorumController id=0] In the new epoch 1, the leader is (none). (org.apache.kafka.controller.QuorumController)
[2024-05-25 15:54:02,983] INFO [MetadataLoader id=0] initializeNewPublishers: the loader is still catching up because we still don't know the high water mark yet. (org.apache.kafka.image.loader.MetadataLoader)
[2024-05-25 15:54:03,039] WARN [RaftManager id=0] Error connecting to node kafka-1:9093 (id: 1 rack: null) (org.apache.kafka.clients.NetworkClient)
java.net.UnknownHostException: kafka-1
    at java.base/java.net.InetAddress$CachedAddresses.get(InetAddress.java:801)
    at java.base/java.net.InetAddress.getAllByName0(InetAddress.java:1533)
    at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1385)
    at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1306)
    at org.apache.kafka.clients.DefaultHostResolver.resolve(DefaultHostResolver.java:27)
    at org.apache.kafka.clients.ClientUtils.resolve(ClientUtils.java:124)
    at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.resolveAddresses(ClusterConnectionStates.java:536)
    at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.currentAddress(ClusterConnectionStates.java:511)
    at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.access$200(ClusterConnectionStates.java:466)
    at org.apache.kafka.clients.ClusterConnectionStates.currentAddress(ClusterConnectionStates.java:173)
    at org.apache.kafka.clients.NetworkClient.initiateConnect(NetworkClient.java:1050)
    at org.apache.kafka.clients.NetworkClient.ready(NetworkClient.java:310)
    at org.apache.kafka.server.util.InterBrokerSendThread.sendRequests(InterBrokerSendThread.java:145)
    at org.apache.kafka.server.util.InterBrokerSendThread.pollOnce(InterBrokerSendThread.java:108)
    at org.apache.kafka.server.util.InterBrokerSendThread.doWork(InterBrokerSendThread.java:136)
    at org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:131)
[2024-05-25 15:54:03,084] INFO [MetadataLoader id=0] initializeNewPublishers: the loader is still catching up because we still don't know the high water mark yet. (org.apache.kafka.image.loader.MetadataLoader)
[2024-05-25 15:54:03,114] WARN [RaftManager id=0] Error connecting to node kafka-1:9093 (id: 1 rack: null) (org.apache.kafka.clients.NetworkClient)
java.net.UnknownHostException: kafka-1
    at java.base/java.net.InetAddress$CachedAddresses.get(InetAddress.java:801)
    at java.base/java.net.InetAddress.getAllByName0(InetAddress.java:1533)
    at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1385)
    at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1306)
    at org.apache.kafka.clients.DefaultHostResolver.resolve(DefaultHostResolver.java:27)
    at org.apache.kafka.clients.ClientUtils.resolve(ClientUtils.java:124)
    at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.resolveAddresses(ClusterConnectionStates.java:536)
    at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.currentAddress(ClusterConnectionStates.java:511)
    at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.access$200(ClusterConnectionStates.java:466)
    at org.apache.kafka.clients.ClusterConnectionStates.currentAddress(ClusterConnectionStates.java:173)
    at org.apache.kafka.clients.NetworkClient.initiateConnect(NetworkClient.java:1050)
    at org.apache.kafka.clients.NetworkClient.ready(NetworkClient.java:310)
    at org.apache.kafka.server.util.InterBrokerSendThread.sendRequests(InterBrokerSendThread.java:145)
    at org.apache.kafka.server.util.InterBrokerSendThread.pollOnce(InterBrokerSendThread.java:108)
    at org.apache.kafka.server.util.InterBrokerSendThread.doWork(InterBrokerSendThread.java:136)
    at org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:131)
[2024-05-25 15:54:03,184] INFO [MetadataLoader id=0] initializeNewPublishers: the loader is still catching up because we still don't know the high water mark yet. (org.apache.kafka.image.loader.MetadataLoader)
[2024-05-25 15:54:03,255] WARN [RaftManager id=0] Error connecting to node kafka-1:9093 (id: 1 rack: null) (org.apache.kafka.clients.NetworkClient)
java.net.UnknownHostException: kafka-1
    at java.base/java.net.InetAddress$CachedAddresses.get(InetAddress.java:801)
    at java.base/java.net.InetAddress.getAllByName0(InetAddress.java:1533)
    at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1385)
    at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1306)
    at org.apache.kafka.clients.DefaultHostResolver.resolve(DefaultHostResolver.java:27)
    at org.apache.kafka.clients.ClientUtils.resolve(ClientUtils.java:124)
    at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.resolveAddresses(ClusterConnectionStates.java:536)
    at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.currentAddress(ClusterConnectionStates.java:511)
    at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.access$200(ClusterConnectionStates.java:466)
    at org.apache.kafka.clients.ClusterConnectionStates.currentAddress(ClusterConnectionStates.java:173)
    at org.apache.kafka.clients.NetworkClient.initiateConnect(NetworkClient.java:1050)
    at org.apache.kafka.clients.NetworkClient.ready(NetworkClient.java:310)
    at org.apache.kafka.server.util.InterBrokerSendThread.sendRequests(InterBrokerSendThread.java:145)
    at org.apache.kafka.server.util.InterBrokerSendThread.pollOnce(InterBrokerSendThread.java:108)
    at org.apache.kafka.server.util.InterBrokerSendThread.doWork(InterBrokerSendThread.java:136)
    at org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:131)
[2024-05-25 15:54:03,284] INFO [MetadataLoader id=0] initializeNewPublishers: the loader is still catching up because we still don't know the high water mark yet. (org.apache.kafka.image.loader.MetadataLoader)
[2024-05-25 15:54:03,385] INFO [MetadataLoader id=0] initializeNewPublishers: the loader is still catching up because we still don't know the high water mark yet. (org.apache.kafka.image.loader.MetadataLoader)
[2024-05-25 15:54:03,486] INFO [MetadataLoader id=0] initializeNewPublishers: the loader is still catching up because we still don't know the high water mark yet. (org.apache.kafka.image.loader.MetadataLoader)
[2024-05-25 15:54:03,495] WARN [RaftManager id=0] Error connecting to node kafka-1:9093 (id: 1 rack: null) (org.apache.kafka.clients.NetworkClient)
java.net.UnknownHostException: kafka-1
    at java.base/java.net.InetAddress$CachedAddresses.get(InetAddress.java:801)
    at java.base/java.net.InetAddress.getAllByName0(InetAddress.java:1533)
    at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1385)
    at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1306)
    at org.apache.kafka.clients.DefaultHostResolver.resolve(DefaultHostResolver.java:27)
    at org.apache.kafka.clients.ClientUtils.resolve(ClientUtils.java:124)
    at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.resolveAddresses(ClusterConnectionStates.java:536)
    at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.currentAddress(ClusterConnectionStates.java:511)
    at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.access$200(ClusterConnectionStates.java:466)
    at org.apache.kafka.clients.ClusterConnectionStates.currentAddress(ClusterConnectionStates.java:173)
    at org.apache.kafka.clients.NetworkClient.initiateConnect(NetworkClient.java:1050)
    at org.apache.kafka.clients.NetworkClient.ready(NetworkClient.java:310)
    at org.apache.kafka.server.util.InterBrokerSendThread.sendRequests(InterBrokerSendThread.java:145)
    at org.apache.kafka.server.util.InterBrokerSendThread.pollOnce(InterBrokerSendThread.java:108)
    at org.apache.kafka.server.util.InterBrokerSendThread.doWork(InterBrokerSendThread.java:136)
    at org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:131)

What do you see instead?

[2024-05-25 15:54:02,961] INFO [RaftManager id=0] Completed transition to CandidateState(localId=0, epoch=1, retries=1, voteStates={0=GRANTED, 1=UNRECORDED}, highWatermark=Optional.empty, electionTimeoutMs=1783) from Unattached(epoch=0, voters=[0, 1], electionTimeoutMs=1418) (org.apache.kafka.raft.QuorumState)
[2024-05-25 15:54:02,965] INFO [QuorumController id=0] In the new epoch 1, the leader is (none). (org.apache.kafka.controller.QuorumController)
[2024-05-25 15:54:02,983] INFO [MetadataLoader id=0] initializeNewPublishers: the loader is still catching up because we still don't know the high water mark yet. (org.apache.kafka.image.loader.MetadataLoader)
[2024-05-25 15:54:03,039] WARN [RaftManager id=0] Error connecting to node kafka-1:9093 (id: 1 rack: null) (org.apache.kafka.clients.NetworkClient)
java.net.UnknownHostException: kafka-1
    at java.base/java.net.InetAddress$CachedAddresses.get(InetAddress.java:801)
    at java.base/java.net.InetAddress.getAllByName0(InetAddress.java:1533)
    at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1385)
    at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1306)
    at org.apache.kafka.clients.DefaultHostResolver.resolve(DefaultHostResolver.java:27)
    at org.apache.kafka.clients.ClientUtils.resolve(ClientUtils.java:124)
    at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.resolveAddresses(ClusterConnectionStates.java:536)
    at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.currentAddress(ClusterConnectionStates.java:511)
    at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.access$200(ClusterConnectionStates.java:466)
    at org.apache.kafka.clients.ClusterConnectionStates.currentAddress(ClusterConnectionStates.java:173)
    at org.apache.kafka.clients.NetworkClient.initiateConnect(NetworkClient.java:1050)
    at org.apache.kafka.clients.NetworkClient.ready(NetworkClient.java:310)
    at org.apache.kafka.server.util.InterBrokerSendThread.sendRequests(InterBrokerSendThread.java:145)
    at org.apache.kafka.server.util.InterBrokerSendThread.pollOnce(InterBrokerSendThread.java:108)
    at org.apache.kafka.server.util.InterBrokerSendThread.doWork(InterBrokerSendThread.java:136)
    at org.apache.kafka.server.util.ShutdownableThread.run(ShutdownableThread.java:131)
[2024-05-25 15:54:03,084] INFO [MetadataLoader id=0] initializeNewPublishers: the loader is still catching up because we still don't know the high water mark yet. (org.apache.kafka.image.loader.MetadataLoader)

Additional information

I also have an issue tracking in helm chart - https://github.com/bitnami/charts/issues/26387

achar007 commented 3 months ago

@carrodher can you help me understand if this is a know issue ? as I see some tests against kafka with Kraft mode tests are failing - https://ge.apache.org/s/tnklt6n7t4ceg/tests/task/:tools:test/details/org.apache.kafka.tools.MetadataQuorumCommandTest/testDescribeQuorumReplicationSuccessful()%5B5%5D?top-execution=1

What's the stable Kafka container image and Kafka helm chart version ? When can we expect a fix for this issue ?

carrodher commented 3 months ago

The issue may not be directly related to the Bitnami container image/Helm chart, but rather to how the application is being utilized, configured in your specific environment, or tied to a specific scenario that is not easy to reproduce on our side.

If you think that's not the case and are interested in contributing a solution, we welcome you to create a pull request. The Bitnami team is excited to review your submission and offer feedback. You can find the contributing guidelines here.

Your contribution will greatly benefit the community. Feel free to reach out if you have any questions or need assistance.

Suppose you have any questions about the application, customizing its content, or technology and infrastructure usage. In that case, we highly recommend that you refer to the forums and user guides provided by the project responsible for the application or technology.

With that said, we'll keep this ticket open until the stale bot automatically closes it, in case someone from the community contributes valuable insights.

github-actions[bot] commented 3 months ago

This Issue has been automatically marked as "stale" because it has not had recent activity (for 15 days). It will be closed if no further activity occurs. Thanks for the feedback.

github-actions[bot] commented 2 months ago

Due to the lack of activity in the last 5 days since it was marked as "stale", we proceed to close this Issue. Do not hesitate to reopen it later if necessary.

Enissay commented 1 month ago

@achar007 I just faced a similar issue, have you had any luck solving it ?