search

apache / camel-kafka-connector

Camel Kafka Connector allows you to use all Camel components as Kafka Connect connectors
https://camel.apache.org
Apache License 2.0
148 stars 99 forks source link

Migrate connectors to kafka 3.6.x plugin discovery #1633

Open davsclaus opened 1 month ago

davsclaus commented 1 month ago

https://kafka.apache.org/documentation.html#connect_plugindiscovery

[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.776 s <<< FAILURE! - in org.apache.camel.kafkaconnector.cassandra.sink.CamelSinkCassandraITCase [ERROR] org.apache.camel.kafkaconnector.cassandra.sink.CamelSinkCassandraITCase Time elapsed: 0.776 s <<< ERROR! org.apache.kafka.connect.errors.ConnectException: One or more plugins are missing ServiceLoader manifests may not be usable with plugin.discovery=service_load: [ classpath io.apicurio.registry.utils.converter.AvroConverter converter undefined classpath io.apicurio.registry.utils.converter.ExtJsonConverter converter undefined classpath io.apicurio.registry.utils.converter.SchemalessConverter converter undefined classpath org.apache.camel.kafkaconnector.CamelSinkConnector sink 4.4.2-SNAPSHOT classpath org.apache.camel.kafkaconnector.CamelSourceConnector source 4.4.2-SNAPSHOT file:/Users/davsclaus/workspace/camel-kafka-connector/connectors/camel-cassandra-sink-kafka-connector/target/classes/ org.apache.camel.kafkaconnector.cassandrasink.CamelCassandrasinkSinkConnector sink 4.4.2-SNAPSHOT classpath org.apache.camel.kafkaconnector.transforms.CamelTypeConverterTransform$Key transformation undefined classpath org.apache.camel.kafkaconnector.transforms.CamelTypeConverterTransform$Value transformation undefined classpath org.apache.camel.kafkaconnector.transforms.ConnectRecordValueToMapTransforms transformation undefined classpath org.apache.camel.kafkaconnector.transforms.FieldsToHeadersTransform$Key transformation undefined classpath org.apache.camel.kafkaconnector.transforms.FieldsToHeadersTransform$Value transformation undefined classpath org.apache.camel.kafkaconnector.transforms.SchemaAndStructToJsonTransform transformation undefined classpath org.apache.camel.kafkaconnector.transforms.SinkPojoToSchemaAndStructTransform transformation undefined classpath org.apache.camel.kafkaconnector.transforms.SourcePojoToSchemaAndStructTransform transformation undefined ] Read the documentation at https://kafka.apache.org/documentation.html#connect_plugindiscovery for instructions on migrating your plugins to take advantage of the performance improvements of service_load mode. To silence this error, set plugin.discovery=hybrid_warn in the worker config. at org.apache.camel.kafkaconnector.cassandra.sink.CamelSinkCassandraITCase.(CamelSinkCassandraITCase.java:45)

davsclaus commented 1 month ago

This works on the older kafka 3.5.x base, so we need to migrate for 3.6.x onwards.

davsclaus commented 4 days ago

We have a hybrid discovery now. But I think it would be good to migrate to new style in the future and to drop hybrid