Upgraded to Kafka 0.11.0.0

[alienscience]

I tried to use this and Kafka keeps bouncing up and down in the docker container:

2017-08-23 13:26:02,379 CRIT Supervisor running as root (no user in config file)
2017-08-23 13:26:02,379 WARN Included extra file "/etc/supervisor/conf.d/kafka.conf" during parsing
2017-08-23 13:26:02,379 WARN Included extra file "/etc/supervisor/conf.d/zookeeper.conf" during parsing
2017-08-23 13:26:02,402 INFO RPC interface 'supervisor' initialized
2017-08-23 13:26:02,403 CRIT Server 'unix_http_server' running without any HTTP authentication checking
2017-08-23 13:26:02,403 INFO supervisord started with pid 1
2017-08-23 13:26:03,404 INFO spawned: 'zookeeper' with pid 10
2017-08-23 13:26:03,405 INFO spawned: 'kafka' with pid 11
2017-08-23 13:26:04,424 INFO success: zookeeper entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2017-08-23 13:26:04,424 INFO success: kafka entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2017-08-23 13:26:04,860 INFO exited: kafka (exit status 1; not expected)
2017-08-23 13:26:05,863 INFO spawned: 'kafka' with pid 289
2017-08-23 13:26:06,978 INFO success: kafka entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2017-08-23 13:26:06,979 INFO exited: kafka (exit status 1; not expected)
... snip ...
2017-08-23 13:34:43,738 INFO success: kafka entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2017-08-23 13:34:43,738 INFO exited: kafka (exit status 1; not expected)
2017-08-23 13:34:44,743 INFO spawned: 'kafka' with pid 30805
2017-08-23 13:34:45,752 INFO success: kafka entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2017-08-23 13:34:45,752 INFO exited: kafka (exit status 1; not expected)
2017-08-23 13:34:46,756 INFO spawned: 'kafka' with pid 31060
2017-08-23 13:34:47,812 INFO success: kafka entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2017-08-23 13:34:47,816 INFO exited: kafka (exit status 1; not expected)
2017-08-23 13:34:48,821 INFO spawned: 'kafka' with pid 31315
2017-08-23 13:34:49,830 INFO success: kafka entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2017-08-23 13:34:49,831 INFO exited: kafka (exit status 1; not expected)
2017-08-23 13:34:50,835 INFO spawned: 'kafka' with pid 31570
2017-08-23 13:34:51,855 INFO success: kafka entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2017-08-23 13:34:51,855 INFO exited: kafka (exit status 1; not expected)
2017-08-23 13:34:52,860 INFO spawned: 'kafka' with pid 31825
2017-08-23 13:34:53,930 INFO success: kafka entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2017-08-23 13:34:53,930 INFO exited: kafka (exit status 1; not expected)
2017-08-23 13:34:54,934 INFO spawned: 'kafka' with pid 32080
2017-08-23 13:34:55,954 INFO success: kafka entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2017-08-23 13:34:55,955 INFO exited: kafka (exit status 1; not expected)
2017-08-23 13:34:56,957 INFO spawned: 'kafka' with pid 32335
2017-08-23 13:34:57,915 INFO exited: kafka (exit status 1; not expected)
2017-08-23 13:34:58,918 INFO spawned: 'kafka' with pid 32590

[kentnek]

I found the problem. It turns out the start_kafka.sh script tried to update some kafka config in $KAFKA_HOME/config/server.properties but somehow rendered the file invalid. I got this while running the script manually in the container:

org.apache.kafka.common.config.ConfigException: Invalid value 0advertised.host.name=kafkaserver for configuration group.
initial.rebalance.delay.ms: Not a number of type INT

The config file indeed contained this line at the end: group.initial.rebalance.delay.ms=0advertised.host.name=kafkaserver

After updating the config and restarting the instance, Kafka (version 11.0.1) started working as expected.

I guess the problem is due to the sed command in start_kafka.sh script, but since I don't have any experience working with sed, can someone update it?

[paambaati]

@alienscience @kentnek You can fix that by changing https://github.com/spotify/docker-kafka/blob/fc8cdbd2e23a5cac21e7138d07ea884b4309c59a/kafka/scripts/start-kafka.sh#L23 to —

echo "\nadvertised.host.name=$ADVERTISED_HOST" >> $KAFKA_HOME/config/server.properties

Notice the leading \n. Repeat for https://github.com/spotify/docker-kafka/blob/fc8cdbd2e23a5cac21e7138d07ea884b4309c59a/kafka/scripts/start-kafka.sh#L31