OneCricketeer
commented
4 years ago You have three containers each exposing the same ports and the same name. If you're using a single machine, what's the point of running multiple brokers?
Closed raginjason closed 4 years ago
OneCricketeer
commented
4 years ago You have three containers each exposing the same ports and the same name. If you're using a single machine, what's the point of running multiple brokers?
raginjason
commented
4 years ago You have three containers each exposing the same ports and the same name. If you're using a single machine, what's the point of running multiple brokers?
I'm not exposing any ports at all actually. Am I missing something here?
OneCricketeer
commented
4 years ago How are you expecting to actually send data to Kafka without exposing ports? Similarly, Zookeeper needs ports and environment variables itself to function well
raginjason
commented
4 years ago How are you expecting to actually send data to Kafka without exposing ports? Similarly, Zookeeper needs ports and environment variables itself to function well
By starting the consumers up in other Docker Compose services along side Kafka and Zk
OneCricketeer
commented
4 years ago That makes sense.
Okay, so other thing is that KAFKA_ADVERTISED_LISTENERS: INSIDE://kafka:9093 is telling each container that it is waiting for a container to connect to kafka:9093... But your services are actually named kafka_{1..3}
sscaling
commented
4 years ago This looks to have been discussed and the issue identified. I don't think the scale parameter will be your friend here because of the way the container names (and subsequently routable hostnames in the bridge network) get created. In the tutorial / readme this works because the README uses your public IP (on the LAN) as the advertised host so both clients inside and outside the docker network can route to the brokers. This is explained in the second use case in the Connectivity Guide. If you don't need / want to expose the kafka port, i've typically seen people just defining multiple kafka services with different names. There are various examples in the closed/open issues.
Closing due to staleness as no further response from OP.
When trying to execute
docker-compose up --no-recreate -d --scale kafka=3it seems that the Kafka containers are stuck printing "waiting for kafka to be ready" to the log. There's a lot going on in the log, but I did notice something that may be a hint:I suspect this is a race condition, but I'm really not sure. Here is my docker-compose.yml and logfiles:
docker-compose.yml:
Docker logfile