Yolean / kubernetes-kafka

Kafka cluster as Kubernetes StatefulSet, plain manifests and config
Apache License 2.0
1.83k stars 738 forks source link

Extract entrypoint command from bin scripts, try native-image zookeeper #311

Closed solsson closed 3 years ago

solsson commented 4 years ago

Use the command generated by Kafka's ./bin/*.sh as entrypoints. Given the same env these scripts simply produce the same command every time, and append args. See #309 and https://github.com/solsson/dockerfiles/blob/master/hooks/build.

Now things like #306 should be done with --override.

solsson commented 4 years ago

I had a very strange init error that isn't reproducible: the symlink operation failed, complaining that the target path exists.

+ '[' '!' -d /var/lib/zookeeper/data ']'
+ '[' -z 1 ']'
+ export ZOOKEEPER_SERVER_ID=2
+ ZOOKEEPER_SERVER_ID=2
+ echo 2
+ tee /var/lib/zookeeper/data/myid
2
+ cp -Lur /etc/kafka-configmap/init.sh /etc/kafka-configmap/log4j.properties /etc/kafka-configmap/zookeeper.properties /etc/kafka/
+ '[' '!' -z 0 ']'
+ '[' '!' -z 3 ']'
+ sed -i 's/^server\./#server./' /etc/kafka/zookeeper.properties
++ seq 0
++ seq 3
+ for N in $(seq $(( $REPLICAS - $PZOO_REPLICAS )))
+ echo server.1=zoo-0.zoo:2888:3888:participant
+ for N in $(seq $(( $REPLICAS - $PZOO_REPLICAS )))
+ echo server.2=zoo-1.zoo:2888:3888:participant
+ for N in $(seq $(( $REPLICAS - $PZOO_REPLICAS )))
+ echo server.3=zoo-2.zoo:2888:3888:participant
+ sed -i 's/server\.2\=[a-z0-9.-]*/server.2=0.0.0.0/' /etc/kafka/zookeeper.properties
+ ln -s /etc/kafka/zookeeper.properties /etc/kafka/zookeeper.properties.scale-3.zoo-1
ln: failed to create symbolic link '/etc/kafka/zookeeper.properties.scale-3.zoo-1': File exists

After delete pod the next pod came up as usual.

solsson commented 3 years ago

To summarize changes from [v6.0.4]():

The ambition was to keep the base folders kafka and zookeeper unchanged, but we had to do some changes to init scripts and container args in order to support an updated nonroot base and the new native base.