How to produce and consume from outside of the docker host ?

[hieutrtr]

Hi Ches,

After start zookeeper and kafka with docker-compose, I have "docker ps" as below

CONTAINER ID        IMAGE                          COMMAND                CREATED             STATUS              PORTS                                         NAMES
5bde6f76246e        hieutrtr/docker-kafka:0.0.1    "/start.sh"            About an hour ago   Up About an hour    7203/tcp, 0.0.0.0:32884->9092/tcp             dockerkafka_kafka_3
be354f1b8cc0        hieutrtr/docker-ubuntu:devel   "/usr/bin/supervisor   About an hour ago   Up About an hour    22/tcp                                        producer1
50d3203af90e        hieutrtr/docker-kafka:0.0.1    "/start.sh"            About an hour ago   Up About an hour    7203/tcp, 0.0.0.0:32883->9092/tcp             dockerkafka_kafka_2
61b285f39615        hieutrtr/docker-kafka:0.0.1    "/start.sh"            2 hours ago         Up 2 hours          7203/tcp, 0.0.0.0:32882->9092/tcp             dockerkafka_kafka_1
20c9c5ccec05        jplock/zookeeper:3.4.6         "/opt/zookeeper/bin/   2 hours ago         Up 2 hours          2888/tcp, 3888/tcp, 0.0.0.0:32881->2181/tcp   dockerkafka_zookeeper_1

But i just can run producer and consumer by command below on the host which docker was installed :

bin/kafka-console-producer.sh --topic test --broker-list $KAFKA_IP:9092

OR

bin/kafka-console-producer.sh --topic test --broker-list $DOCKER_HOST:32882

Could you please show me the way to allow producer and consumer can be call from another host.

Thanks, HieuTrTr

[hieutrtr]

I run producer from my localhost to kafka container on docker host and i got error like this :

➜  kafka_2.9.1-0.8.2.1: bin/kafka-console-producer.sh --topic test --broker-list $DOCKER_HOST:32884
[2015-08-31 06:55:15,450] WARN Property topic is not valid (kafka.utils.VerifiableProperties)
to
[2015-08-31 06:55:20,214] WARN Failed to send producer request with correlation id 2 to broker 1 with data for partitions [test,0] (kafka.producer.async.DefaultEventHandler)
java.nio.channels.ClosedChannelException
    at kafka.network.BlockingChannel.send(BlockingChannel.scala:100)
    at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:73)
    at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:72)
    at kafka.producer.SyncProducer$$anonfun$send$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(SyncProducer.scala:103)
    at kafka.producer.SyncProducer$$anonfun$send$1$$anonfun$apply$mcV$sp$1.apply(SyncProducer.scala:103)
    at kafka.producer.SyncProducer$$anonfun$send$1$$anonfun$apply$mcV$sp$1.apply(SyncProducer.scala:103)

[ches]

Hi,

Sorry for the delay in answering. This section of the README addresses your question—if you start the Kafka containers that way you should be able to connect from a client running directly on your host system. The KAFKA_ADVERTISED_HOST_NAME is essential. Of course you'll need to adjust port mappings if you'd like to run multiple brokers, and be sure to also set unique KAFKA_BROKER_ID values.

If you're still having trouble, please paste your docker-compose.yml.