banzaicloud / koperator

Oh no! Yet another Apache Kafka operator for Kubernetes
Apache License 2.0
787 stars 198 forks source link

unable to bring up kafka v1.0.2 cluster #172

Closed rockmenjack closed 4 years ago

rockmenjack commented 5 years ago

Describe the bug operator v0.7.0 using clusterImage: "wurstmeister/kafka:2.11-1.0.2" will result in error:

2019-10-24T03:46:14.297Z        ERROR   controller-runtime.controller   Reconciler error        {"controller": "kafkacluster", "request": "osp/kafka", "error": "could not describe cluster wide broker config: Broker id must be an integer, but it is: ", "errorVerbose": "Broker id must be an integer, but it is: \ncould not describe cluster wide broker config\ngithub.com/banzaicloud/kafka-operator/pkg/resources/kafka.(*Reconciler).reconcileClusterWideDynamicConfig\n\t/workspace/pkg/resources/kafka/kafka.go:392\ngithub.com/banzaicloud/kafka-operator/pkg/resources/kafka.(*Reconciler).Reconcile\n\t/workspace/pkg/resources/kafka/kafka.go:284\ngithub.com/banzaicloud/kafka-operator/controllers.(*KafkaClusterReconciler).Reconcile\n\t/workspace/controllers/kafkacluster_controller.go:114\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.2.0/pkg/internal/controller/controller.go:216\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.2.0/pkg/internal/controller/controller.go:192\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.2.0/pkg/internal/controller/controller.go:171\nk8s.io/apimachinery/pkg/util/wait.JitterUntil.func1\n\t/go/pkg/mod/k8s.io/apimachinery@v0.0.0-20190404173353-6a84e37a896d/pkg/util/wait/wait.go:152\nk8s.io/apimachinery/pkg/util/wait.JitterUntil\n\t/go/pkg/mod/k8s.io/apimachinery@v0.0.0-20190404173353-6a84e37a896d/pkg/util/wait/wait.go:153\nk8s.io/apimachinery/pkg/util/wait.Until\n\t/go/pkg/mod/k8s.io/apimachinery@v0.0.0-20190404173353-6a84e37a896d/pkg/util/wait/wait.go:88\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1357"}

the cause seems coming from : kafka, I am suspecting API change, so github.com/Shopify/sarama does not work for pre-2.x.x kafka any more...

baluchicken commented 5 years ago

Thanks for reporting this. The operator supports Kafka from version 2.0 We are using Kafka feature called dynamic configuration, which is introduced in version 1.1. That caused the error for you. Please check the official kafka documentation: https://kafka.apache.org/documentation/#dynamicbrokerconfigs You can try to use 1.1 but we are also using a Cruise Control which is built for Kafka version 2.0+. I am going to fix the documentation and include the supported Kafka version.

baluchicken commented 4 years ago

We included the supported Kafka versions to the README. We hope we answered your question, so closing this. Please contact us here or on our community slack if you need more help.