search

conduktor / kafka-stack-docker-compose

docker compose files to create a fully working kafka stack
Apache License 2.0
3.24k stars 1.25k forks source link

Unavailability of DOCKER_HOST_IP gives invalid receive exception #45

Closed codeasashu closed 2 years ago

codeasashu commented 5 years ago

I forgot to set the DOCKER_HOST_IP environment variable before launching the stack by docker-compsoe -f zk-single-kafka-single.yml up -d. It did launched and everything seemed alright until I wrote a consumer in PHP using librdkafka and php-rdkafka.

The php code wasn't able to connect to kafka instance. Upon checking the kafka logs in the stack docker-compose -f zk-single-kafka-single.yml I see this:

[2019-02-02 14:10:13,267] WARN [SocketServer brokerId=1] Unexpected error from /172.18.0.1; closing connection (org.apache.kafka.common.network.Selector)
org.apache.kafka.common.network.InvalidReceiveException: Invalid receive (size = -196097)
    at org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:102)
    at org.apache.kafka.common.network.KafkaChannel.receive(KafkaChannel.java:390)
    at org.apache.kafka.common.network.KafkaChannel.read(KafkaChannel.java:351)
    at org.apache.kafka.common.network.Selector.attemptRead(Selector.java:609)
    at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:541)
    at org.apache.kafka.common.network.Selector.poll(Selector.java:467)
    at kafka.network.Processor.poll(SocketServer.scala:689)
    at kafka.network.Processor.run(SocketServer.scala:594)
    at java.lang.Thread.run(Thread.java:748)

I couldn't figure this out for couple of hours until I stopped the stack, set the DOCKER_HOST_IP env var and starting the stack up again. It worked perfectly after.

IMO, you can provide a entrypoint script which can halt the startup if env var is unavailable or if possible can use the DNS if provided

avandermeyden commented 4 years ago

I am running this on an Ubuntu VM, which is using DHCP, so the IP isn't predictable. I added the following to my bash startup script (you could use ~/profile - I used /etc/bash.bashrc so it worked on all users) to export the DOCKER_HOST_IP:

# set DOCKER_HOST_IP
export DOCKER_HOST_IP=$(ip route get 1 | awk '{print $(NF-2);exit}')

This exported the environment variable, but the background start still didn't work properly until I did this:

sudo --preserve-env=DOCKER_HOST_IP docker-compose up -d
simplesteph commented 2 years ago

Closing this as inactive