search

confluentinc / cp-helm-charts

The Confluent Platform Helm charts enable you to deploy Confluent Platform services on Kubernetes for development, test, and proof of concept environments.
https://cnfl.io/getting-started-kafka-kubernetes
Apache License 2.0
787 stars 846 forks source link

cp-helm-chart 5.3.0 bug? #311

Open emy-lee opened 5 years ago

emy-lee commented 5 years ago

I have installed confluent 5.3.0 via helm but I get errors. I have created a custom values.yaml file to use cp-kafka instead of cp-enterprise-kafka, 5.3.0 instead of 5.2.2 and disabled cp-control-center.

$ kubectl get pods
NAME                                               READY   STATUS             RESTARTS   AGE
emiconfluent-cp-kafka-0                            1/2     CrashLoopBackOff   3          4m7s
emiconfluent-cp-kafka-1                            1/2     CrashLoopBackOff   3          2m28s
emiconfluent-cp-kafka-2                            1/2     Error              3          2m12s
emiconfluent-cp-kafka-connect-c5946c74f-2hj5b      1/2     Error              3          4m7s
emiconfluent-cp-kafka-rest-59bdf6b45-tzldx         2/2     Running            3          4m7s
emiconfluent-cp-ksql-server-ddfcfb4dd-shkhw        2/2     Running            0          4m7s
emiconfluent-cp-schema-registry-5cf8f864cf-2hcjz   1/2     CrashLoopBackOff   3          4m7s
emiconfluent-cp-zookeeper-0                        2/2     Running            1          4m6s
emiconfluent-cp-zookeeper-1                        2/2     Running            0          2m24s
emiconfluent-cp-zookeeper-2                        2/2     Running            0          2m7s
kafka-client                                       1/1     Running            0          2m20s
zookeeper-client                                   1/1     Running            0          2m25s

I noticed that many people have this problem.

kubectl logs --follow emiconfluent-cp-kafka-0 -c cp-kafka-broker
[2019-08-02 10:58:08,484] ERROR [KafkaServer id=0] Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
org.apache.kafka.common.KafkaException: io.confluent.metrics.reporter.ConfluentMetricsReporter ClassNotFoundException exception occurred
       at org.apache.kafka.common.config.AbstractConfig.getConfiguredInstances(AbstractConfig.java:425)
       at kafka.server.DynamicMetricsReporters.createReporters(DynamicBrokerConfig.scala:744)
       at kafka.server.DynamicMetricsReporters.<init>(DynamicBrokerConfig.scala:693)
       at kafka.server.DynamicBrokerConfig.addReconfigurables(DynamicBrokerConfig.scala:227)
       at kafka.server.KafkaServer.startup(KafkaServer.scala:318)
       at io.confluent.support.metrics.SupportedServerStartable.startup(SupportedServerStartable.java:114)
       at io.confluent.support.metrics.SupportedKafka.main(SupportedKafka.java:66)
Caused by: java.lang.ClassNotFoundException: io.confluent.metrics.reporter.ConfluentMetricsReporter
       at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
       at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
       at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
       at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
       at java.lang.Class.forName0(Native Method)
       at java.lang.Class.forName(Class.java:348)
       at org.apache.kafka.common.utils.Utils.loadClass(Utils.java:348)
       at org.apache.kafka.common.utils.Utils.newInstance(Utils.java:337)
       at org.apache.kafka.common.config.AbstractConfig.getConfiguredInstances(AbstractConfig.java:423)
       ... 6 more

If I edit the template ( https://github.com/confluentinc/cp-helm-charts/blob/v5.2.2-1/charts/cp-kafka/templates/statefulset.yaml ) and comment the rows from the 113 to 117, then kafka works. Is this a bug? Is there a way to unset those vars without editing the template ( maybe using some flags )?

wheresalice commented 5 years ago

This is probably a duplicate of #305 , but this issue adds more details

gAmUssA commented 5 years ago

@emy-lee try to remove config on KAFKA_METRIC_REPORTERS.

https://github.com/confluentinc/cp-helm-charts/blob/51ae55d90b479a763d6a9230fa6c4d0ac548c66d/charts/cp-kafka/templates/statefulset.yaml#L114

I think we need to make configurable

gAmUssA commented 5 years ago

@emy-lee BTW, I don't think its bug per se (because you want to use «different image», default image is cp-enterprise in oder to control center to work). But I'm sure we can improve user experience and documentation in this regard. If you want to make this configurable - like «if you want to run only community versions do that» - feel free to send a PR. I can help working on this

mjschmidt commented 5 years ago

@gAmUssA If we can't replace the enterprise images with the community images and having things work because images wake up and look for non existent classes, but there is also no configuration to turn that class it is looking for off. I'd argue that is a bug.

mjschmidt commented 5 years ago

Especially since it use to work. @emy-lee as a work around, if you have the older helm chart package you can just uptic the old values file there. However, it looks like confluent may have not been versioning this helm chart package since I don't see any branches for versions and I don't think that the charts are hosted in any versioned helm repo. If I am wrong please let me know, because right now I just have an old on on my file system. Which is not sustainable (:

gAmUssA commented 5 years ago

@mjschmidt I will take a look at this next week. If anyone wants to propose a configuration switch (ce with c3 or cc without c3) I'm willing to help out. Cheers Vik

omanderson commented 5 years ago

@mjschmidt I will take a look at this next week. If anyone wants to propose a configuration switch (ce with c3 or cc without c3) I'm willing to help out. Cheers Vik

Hello Vik, any update about this issue?

gAmUssA commented 5 years ago

@omanderson was busy lunching https://kafka-tutorials.confluent.io last week. Hopefully will fix this week. But if you have idea how to do it - go ahead, I will review and merge. Thank you

omanderson commented 5 years ago

@omanderson was busy lunching https://kafka-tutorials.confluent.io last week. Hopefully will fix this week. But if you have idea how to do it - go ahead, I will review and merge. Thank you

I would like to do it, but I don't know how to do it Thank you :)

gAmUssA commented 5 years ago

@omanderson

It's not difficult. You can use Helm's flow controls structures https://github.com/helm/helm/blob/master/docs/chart_template_guide/control_structures.md

_IF Confluent Enterprise disabled use cp-kafka image and don't configure METRICS_REPORTER and so on._

Here is an example how it's done for ksql headless mode

xxmaestroxx commented 4 years ago

@gAmUssA

There are already multiple pull requests which try to solve this issue: https://github.com/confluentinc/cp-helm-charts/pull/436 and https://github.com/confluentinc/cp-helm-charts/pull/378 can you somehow move some focus/help on it?

Would be very helpful whenever you just want to install kafka + zookeeper without the rest.

gAmUssA commented 4 years ago

@xxmaestroxx I commented on those issues. we need close one and merge another. They look like duplicates. Will see who will respond first

leshibily commented 3 years ago

@gAmUssA I am really sad to see none of the ticket related to this issue got closed even after a year.