baluchicken
commented
4 years ago Hi, Thanks for opening this ticket. The Operator relies 100% on Kafka in case of ZkPath maintenance. Since Kafka itself does not remove all ZkPath, the operator does not remove it as well. Although, we are considering to do some cleanup when a different zkPath is used than root.
dobrerazvan
Describe the bug When a cluster is deleted, zkPath zk node is not deleted when the kafka cluster is deleted.
Steps to reproduce the issue:
Expected behavior My expectation is that operator will remove the zkPath node on kafka cluster termination.
Screenshots ▶ kubectl apply -f helm/kafka-k8s/pipeline-kafka23-1disk.yaml -n pipeline-kafka23-1disk kafkacluster.kafka.banzaicloud.io/pipeline-kafka23-1disk created ▶ kubectl get pods -n pipeline-kafka23-1disk NAME READY STATUS RESTARTS AGE envoy-6bddd47b54-hj6hb 1/1 Running 0 21m envoy-6bddd47b54-j28mp 1/1 Running 0 21m envoy-6bddd47b54-tfdpc 1/1 Running 0 21m envoy-6bddd47b54-tmqk6 1/1 Running 0 21m envoy-6bddd47b54-vnr5p 1/1 Running 0 21m envoy-6bddd47b54-xnpwm 1/1 Running 0 21m pipeline-kafka23-1disk-cruisecontrol-5c46d95ff6-gxtjh 1/1 Running 0 2m29s pipeline-kafka23-1diskbf5bc 1/1 Running 0 16m pipeline-kafka23-1diskbt5tv 1/1 Running 0 21m pipeline-kafka23-1diskdzmzv 1/1 Running 0 16m pipeline-kafka23-1diskfp6l9 1/1 Running 0 16m pipeline-kafka23-1diskfxqjp 1/1 Running 0 16m pipeline-kafka23-1diskx7t7s 1/1 Running 0 16m ▶ zkCli -server localhost:2181 [zk: localhost:2181(CONNECTED) 6] ls /pipeline-kafka23-1disk/brokers/ids [100, 200, 101, 300, 201, 301] [zk: localhost:2181(CONNECTED) 0] get /pipeline-kafka23-1disk/brokers/ids/100 {"listener_security_protocol_map":{"EXTERNAL":"PLAINTEXT","INTERNAL":"PLAINTEXT","CONTROLLER":"PLAINTEXT"},"endpoints":["EXTERNAL://aa22b91b3443511ea9e090e4e6d5456b-8849a15828064e76.elb.us-east-1.amazonaws.com:19190","INTERNAL://pipeline-kafka23-1disk-100.pipeline-kafka23-1disk-headless.pipeline-kafka23-1disk.svc.cluster.local:29092","CONTROLLER://pipeline-kafka23-1disk-100.pipeline-kafka23-1disk-headless.pipeline-kafka23-1disk.svc.cluster.local:29093"],"rack":"us-east-1,us-east-1a","jmx_port":-1,"host":"aa22b91b3443511ea9e090e4e6d5456b-8849a15828064e76.elb.us-east-1.amazonaws.com","timestamp":"1580481060697","port":19190,"version":4} ▶ kubectl delete -f helm/kafka-k8s/pipeline-kafka23-1disk.yaml -n pipeline-kafka23-1disk kafkacluster.kafka.banzaicloud.io "pipeline-kafka23-1disk" deleted ▶ kubectl get pods -n pipeline-kafka23-1disk No resources found in pipeline-kafka23-1disk namespace. ▶ zkCli -server localhost:2181 Connecting to localhost:2181 Welcome to ZooKeeper! JLine support is enabled [zk: localhost:2181(CONNECTING) 0] [zk: localhost:2181(CONNECTED) 2] ls /pipeline-kafka23-1disk [cluster, controller_epoch, brokers, CruiseControlBrokerList, admin, isr_change_notification, consumers, log_dir_event_notification, latest_producer_id_block, config]
-> my expectation was that /pipeline-kafka23-1disk would be gone at this moment.
Additional context