Cahb
commented
1 month ago Hi @ncalad,
Thanks for opening an issue, and i'm sorry you're having issues;
The reason why kafka / zook startup are failing is due to missed env vars / config params; Could you please try the following steps and tell me if they work for you?
# Create a new network:
docker network create kafka_network
# Create zookeeper container, attach to kafka network, assign static DNS name <zookeeper> to this container within kafka_network:
docker run -d --network=kafka_network --name zookeeper \
-p 2181:2181 \
-e ZOOKEEPER_CLIENT_PORT=2181 \
zookeeper:latest
# Create Kafka container, attach to kafka network, assign static DNS name <kafka> to this container within kafka_network:
docker run -d --network=kafka_network --name kafka -p 9092:9092 \
-e KAFKA_LISTENER_SECURITY_PROTOCOL_MAP=PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT \
-e KAFKA_INTER_BROKER_LISTENER_NAME=PLAINTEXT \
-e KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR=1 \
-e KAFKA_BROKER_ID=1 \
-e KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181 \
-e KAFKA_LISTENERS=PLAINTEXT://0.0.0.0:9092 \
-e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://kafka:9092 wurstmeister/kafka
...
# Launch bash inside container <kafka>
docker exec -it kafka bash
...
<from inside docker container execute the following commands>:
# Create a test topic, 3 partitions name 'test_topic'
kafka-topics.sh --bootstrap-server localhost:9092 --topic test_topic --create --partitions 3 --replication-factor 1
# Try to _list_ created partitions (should be only one partition we created):
kafka-topics.sh --bootstrap-server localhost:9092 --listAfter all that, you should see something like the following output:
...
root@a1fe950e039a:/# kafka-topics.sh --bootstrap-server localhost:9092 --topic test_topic --create --partitions 3 --replication-factor 1
WARNING: Due to limitations in metric names, topics with a period ('.') or underscore ('_') could collide. To avoid issues it is best to use either, but not both.
Created topic test_topic.
root@a1fe950e039a:/# kafka-topics.sh --bootstrap-server localhost:9092 --list
test_topic
...P.S. in case if these commands do not work, please do not hesitate and write me back; I'll try my best to help;
ncalad
We cloned the repo and ran make all. zookeeper is running on port 2818.
docker run -d --name zookeeper -p 2181:2181 zookeeper:latest docker run -d --name kafka -p 9092:9092 -e KAFKA_ZOOKEEPER_CONNECT=localhost:2181 -e KAFKA_LISTENERS=PLAINTEXT://0.0.0.0:9092 -e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://localhost:9092 wurstmeister/kafka
When we run this command, the kafka container runs and then exits:
[2024-09-23 18:56:28,133] INFO Session: 0x0 closed (org.apache.zookeeper.ZooKeeper) [2024-09-23 18:56:28,134] INFO EventThread shut down for session: 0x0 (org.apache.zookeeper.ClientCnxn) [2024-09-23 18:56:28,138] INFO [ZooKeeperClient Kafka server] Closed. (kafka.zookeeper.ZooKeeperClient) [2024-09-23 18:56:28,143] ERROR Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer) kafka.zookeeper.ZooKeeperClientTimeoutException: Timed out waiting for connection while in state: CONNECTING at kafka.zookeeper.ZooKeeperClient.waitUntilConnected(ZooKeeperClient.scala:271) at kafka.zookeeper.ZooKeeperClient.(ZooKeeperClient.scala:125)
at kafka.zk.KafkaZkClient$.apply(KafkaZkClient.scala:1948)
at kafka.server.KafkaServer.createZkClient$1(KafkaServer.scala:431)
at kafka.server.KafkaServer.initZkClient(KafkaServer.scala:456)
at kafka.server.KafkaServer.startup(KafkaServer.scala:191)
at kafka.Kafka$.main(Kafka.scala:109)
at kafka.Kafka.main(Kafka.scala)
[2024-09-23 18:56:28,148] INFO shutting down (kafka.server.KafkaServer)
[2024-09-23 18:56:28,156] INFO App info kafka.server for -1 unregistered (org.apache.kafka.common.utils.AppInfoParser)
[2024-09-23 18:56:28,161] INFO shut down completed (kafka.server.KafkaServer)
[2024-09-23 18:56:28,161] ERROR Exiting Kafka. (kafka.Kafka$)
[2024-09-23 18:56:28,165] INFO shutting down (kafka.server.KafkaServer)
Are we missing a step?
Thank you