search

confluentinc / cp-helm-charts

The Confluent Platform Helm charts enable you to deploy Confluent Platform services on Kubernetes for development, test, and proof of concept environments.
https://cnfl.io/getting-started-kafka-kubernetes
Apache License 2.0
788 stars 847 forks source link

Helm chart kafka connect installation is Missing InfluxDBSinkConnector #505

Open ghost opened 3 years ago

ghost commented 3 years ago

After installing kafka connect helm chart in kubernetes cluster with cp-helm-charts\charts\cp-kafka-connect (6.0.1-ccs), I am getting this error message "{"error_code":500,"message":"Failed to find any class that implements Connector and which name matches io.confluent.influxdb.InfluxDBSinkConnector," Is there any other version of kafka connect helm chart include the io.confluent.influxdb.InfluxDBSinkConnector as in the confluent doc: https://docs.confluent.io/kafka-connect-influxdb/current/influx-db-sink-connector/index.html

ducminhle commented 3 years ago

I think you should build a new Docker image Kafka-connect and install this plugin, and change the Kafka connect image in value.yaml file to use your Docker image built.

confluent-hub install confluentinc/kafka-connect-influxdb:<VERSION>

https://www.confluent.io/hub/confluentinc/kafka-connect-influxdb

ghost commented 3 years ago

Thank you for the reply. I did try to run this install command from the confluent-hub container, it gets installed successfully /usr/share/confluent-hub-components/confluentinc-kafka-connect-influxdb in all the workers, which as plugin.path=/usr/share/java,/usr/share/confluent-hub-components, but still I am getting the same error when I post the connector.json to local rest endpoint.

hoongeun commented 3 years ago

I spent several days about this issue.

In other examples, they installed connect before it start. https://github.com/confluentinc/demo-scene/blob/fdd80c2043c1cc55f45833aa76d4e1abeb740ced/kafka-connect-zero-to-hero/docker-compose.yml#L89-L101

But this chart doesn't https://github.com/confluentinc/cp-helm-charts/blob/c6874284a0141ea6b70b70c9238a7ce09af2920c/charts/cp-kafka-connect/templates/deployment.yaml#L106-L112

OneCricketeer commented 3 years ago

The image doesn't come with the Influx connector

I did try to run this install command from the confluent-hub container

You need to restart the Connect server to load new plugins (which would cause the pod to die)

You therefore need to install before the container starts, for example with your own image, or as an overridden command, as shown