search

Yolean / kubernetes-kafka

Kafka cluster as Kubernetes StatefulSet, plain manifests and config
Apache License 2.0
1.83k stars 738 forks source link

Quickstart is broken (v6.0.3) #346

Open austinorth opened 2 years ago

austinorth commented 2 years ago

Environment Details

Problem 1

The commands in the Quickstart section of the readme do not successfully spin up a Kafka cluster. kubectl create namespace kafka obviously works by itself, but in zsh, I get the following error when running the kubectl apply -k github.com/Yolean/kubernetes-kafka/variants/dev-small/?ref=v6.0.3 command:

❯ k apply -k github.com/Yolean/kubernetes-kafka/variants/dev-small/?ref=v6.0.3
zsh: no matches found: github.com/Yolean/kubernetes-kafka/variants/dev-small/?ref=v6.0.3

Solution to Problem 1

Adding quotes around the URL allows zsh to interpret the command correctly.

❯ k apply -k "github.com/Yolean/kubernetes-kafka/variants/dev-small/?ref=v6.0.3"
role.rbac.authorization.k8s.io/pod-labler created
clusterrole.rbac.authorization.k8s.io/node-reader created
rolebinding.rbac.authorization.k8s.io/kafka-pod-labler created
clusterrolebinding.rbac.authorization.k8s.io/kafka-node-reader created
configmap/broker-config created
configmap/zookeeper-config created
service/bootstrap created
service/broker created
service/pzoo created
service/zoo created
service/zookeeper created
statefulset.apps/kafka created
statefulset.apps/pzoo created
statefulset.apps/zoo created

However, there is a separate problem that occurs with config itself for the dev-small variant.

Problem 2

After the config is successfully applied, if you check the pods, you'll see that the Zookeeper pod is stuck in an unready state, which causes the Kafka pod to be stuck in a restart loop.

❯ kubectl get pods -n kafka
NAME      READY   STATUS    RESTARTS   AGE
kafka-0   0/1     Running   2          36s
pzoo-0    0/1     Running   0          36s

This appears to be due to a failing readiness probe

❯ kubectl describe pod pzoo-0 -n kafka | grep Readiness
    Readiness:  exec [/bin/sh -c [ "imok" = "$(echo ruok | nc -w 1 -q 1 127.0.0.1 2181)" ]] delay=0s timeout=1s period=10s #success=1 #failure=3
  Warning  Unhealthy  27s (x19 over 3m27s)  kubelet            Readiness probe failed:

Not sure how to resolve this one. Hoping someone has some insight! Would love to use this repo for Kafka development!

yhjhoo commented 1 year ago

I got the same, have you resolved this?