Hello
I have been using ches/kafka with docker compose from October last year without problem. Recently I created some new topics but any attempt to produce/consume with those topics fails, although my program still appears to work with the old topics.
In an attempt to investigate I created a new docker-compose.yml with just zookeeper and kafka using new file space. When I run the producer/consumer test I get the same problem, here is some interaction and output;
$ docker-compose ps
Name Command State Ports
kafka_kafka_1 /start.sh Up 7203/tcp, 0.0.0.0:9092->9092/tcp
kafka_zookeeper_1 /docker-entrypoint.sh zkSe ... Up 0.0.0.0:2181->2181/tcp, 2888/tcp, 3888/tcp
$ docker-compose run --rm kafka kafka-console-producer.sh --topic top1 --broker-list kafka:9092
Starting kafka_zookeeper_1 ... done
hello
[2018-04-11 14:09:04,661] ERROR Error when sending message to topic top1 with key: null, value: 5 bytes with error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
org.apache.kafka.common.errors.TimeoutException: Failed to update metadata after 60000 ms.
If I create zookeeper and kafka from the command line it works
docker run -d --name zookeeper --network test-net zookeeper:3.4
docker run -d --name kafka --network test-net --env ZOOKEEPER_IP=zookeeper ches/kafka
docker run --rm --network test-net ches/kafka kafka-topics.sh --create --topic test --replication-factor 1 --partitions 1 --zookeeper zookeeper:2181
docker run --rm --interactive --network test-net ches/kafka kafka-console-producer.sh --topic test --broker-list kafka:9092
in a separate terminal
docker run --rm --network test-net ches/kafka kafka-console-consumer.sh --topic test --from-beginning --bootstrap-server kafka:9092
Hello I have been using ches/kafka with docker compose from October last year without problem. Recently I created some new topics but any attempt to produce/consume with those topics fails, although my program still appears to work with the old topics.
In an attempt to investigate I created a new docker-compose.yml with just zookeeper and kafka using new file space. When I run the producer/consumer test I get the same problem, here is some interaction and output; $ docker-compose ps Name Command State Ports
kafka_kafka_1 /start.sh Up 7203/tcp, 0.0.0.0:9092->9092/tcp kafka_zookeeper_1 /docker-entrypoint.sh zkSe ... Up 0.0.0.0:2181->2181/tcp, 2888/tcp, 3888/tcp
$ docker-compose run --rm kafka kafka-topics.sh --list --zookeeper zookeeper:2181 Starting kafka_zookeeper_1 ... done
$ docker-compose run --rm kafka kafka-topics.sh --create --topic top1 --replication-factor 1 --partitions 1 --zookeeper zookeeper:2181 Starting kafka_zookeeper_1 ... done Created topic "top1".
$ docker-compose run --rm kafka kafka-topics.sh --list --zookeeper zookeeper:2181 Starting kafka_zookeeper_1 ... done top1
$ docker-compose run --rm kafka kafka-console-producer.sh --topic top1 --broker-list kafka:9092 Starting kafka_zookeeper_1 ... done hello [2018-04-11 14:09:04,661] ERROR Error when sending message to topic top1 with key: null, value: 5 bytes with error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback) org.apache.kafka.common.errors.TimeoutException: Failed to update metadata after 60000 ms.
If I create zookeeper and kafka from the command line it works
docker run -d --name zookeeper --network test-net zookeeper:3.4 docker run -d --name kafka --network test-net --env ZOOKEEPER_IP=zookeeper ches/kafka docker run --rm --network test-net ches/kafka kafka-topics.sh --create --topic test --replication-factor 1 --partitions 1 --zookeeper zookeeper:2181 docker run --rm --interactive --network test-net ches/kafka kafka-console-producer.sh --topic test --broker-list kafka:9092
This works fine.
This is my docker-compose file
version: "2" networks: kafka-net:
services: zookeeper: image: zookeeper:3.4 hostname: zookeeper networks:
kafka: image: ches/kafka hostname: kafka links:
Sorry to bother you but I have spent a few days on this and not really making any progress.
Thanks Diane