Closed rmoff closed 4 years ago
Same here. I don't code in Java so I've been struggling to pinpoint the exact problem, but I've found that these four work alright —
[2020-05-25 15:44:04,256] INFO Added alias 'ChangeTopicCase' to plugin 'com.github.jcustenborder.kafka.connect.transform.common.ChangeTopicCase' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 15:44:04,257] INFO Added alias 'HeaderToField' to plugin 'com.github.jcustenborder.kafka.connect.transform.common.HeaderToField' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 15:44:04,257] INFO Added alias 'SetMaximumPrecision' to plugin 'com.github.jcustenborder.kafka.connect.transform.common.SetMaximumPrecision' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 15:44:04,258] INFO Added alias 'TimestampNow' to plugin 'com.github.jcustenborder.kafka.connect.transform.common.TimestampNow' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
Apparently one of them implements Transformation
, and three others implement BaseKeyValueTransformation
, which is nowhere to be found, and yet, those four transformations work 🤷🏼♂️.
@volodymyr-kushnir How did you install this? You need to ensure that the dependencies are there as well. If you copied a single jar you're missing all of the dependencies. This is the easiest way to is to use the Confluent Hub
That's what I did. Here's my Dockerfile —
FROM confluentinc/cp-kafka-connect-base:5.5.0
RUN confluent-hub install --no-prompt jcustenborder/kafka-connect-transform-common:latest
I tried different versions instead of "latest" and then I just gave up and tried this instead —
"transforms": "renameFields",
"transforms.renameFields.type": "org.apache.kafka.connect.transforms.ReplaceField$Value",
"transforms.renameFields.renames": "renameThis:rename_this",
— and it still said that "Class could not be found". Then I downgraded to confluentinc/cp-kafka-connect-base:5.0.0 and it still won't work.
Let me give it a look.
Thanks. Maybe this will help — I see it adding the plugins and aliases, but for some reason it omits adding the aliases for certain plugins, not only yours, but also its own (ReplaceField) as well. I guess I just have to figure out the last version of Kafka Connect where all of these would work 😔.
[2020-05-25 16:48:14,741] INFO Scanning for plugin classes. This might take a moment ... (org.apache.kafka.connect.cli.ConnectDistributed)
[2020-05-25 16:48:14,790] INFO Loading plugin from: /usr/share/confluent-hub-components/jcustenborder-kafka-connect-transform-common (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,426] INFO Registered loader: PluginClassLoader{pluginLocation=file:/usr/share/confluent-hub-components/jcustenborder-kafka-connect-transform-common/} (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,426] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.HeaderToField' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,426] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.BytesToString$Value' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,426] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.ToJSON$Key' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,426] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.PatternFilter$Key' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,426] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.ToLong$Key' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,427] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.BytesToString$Key' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,427] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.ExtractTimestamp$Value' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,427] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.TopicNameToField$Key' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,427] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.ChangeCase$Value' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,427] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.PatternFilter$Value' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,427] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.SetMaximumPrecision$Key' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,427] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.PatternRename$Value' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,427] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.SetMaximumPrecision' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,427] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.ExtractNestedField$Value' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,427] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.TopicNameToField$Value' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,428] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.HeaderToField$Key' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,428] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.ChangeTopicCase' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,428] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.TimestampNow' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,428] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.SchemaNameToTopic$Key' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,428] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.NormalizeSchema$Key' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,428] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.ToLong$Value' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,428] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.PatternRename$Key' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,429] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.SetNull$Value' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,429] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.ExtractNestedField$Key' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,429] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.ChangeCase$Key' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,429] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.SetMaximumPrecision$Value' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,429] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.NormalizeSchema$Value' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,430] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.ExtractTimestamp$Key' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,430] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.HeaderToField$Value' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,430] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.SetNull$Key' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,430] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.ToJSON$Value' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,430] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.TimestampNowField$Value' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,430] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.SchemaNameToTopic$Value' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,431] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.TimestampNowField$Key' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,431] INFO Added plugin 'org.apache.kafka.connect.connector.policy.AllConnectorClientConfigOverridePolicy' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,431] INFO Added plugin 'org.apache.kafka.connect.connector.policy.PrincipalConnectorClientConfigOverridePolicy' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,432] INFO Added plugin 'org.apache.kafka.connect.connector.policy.NoneConnectorClientConfigOverridePolicy' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,152] INFO Registered loader: sun.misc.Launcher$AppClassLoader@764c12b6 (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,152] INFO Added plugin 'org.apache.kafka.connect.mirror.MirrorSourceConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,152] INFO Added plugin 'io.confluent.connect.jdbc.JdbcSinkConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,153] INFO Added plugin 'org.apache.kafka.connect.file.FileStreamSinkConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,153] INFO Added plugin 'io.confluent.connect.jdbc.JdbcSourceConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,153] INFO Added plugin 'org.apache.kafka.connect.file.FileStreamSourceConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,153] INFO Added plugin 'org.apache.kafka.connect.tools.MockSourceConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,153] INFO Added plugin 'org.apache.kafka.connect.tools.SchemaSourceConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,153] INFO Added plugin 'org.apache.kafka.connect.mirror.MirrorCheckpointConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,153] INFO Added plugin 'org.apache.kafka.connect.tools.VerifiableSinkConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,153] INFO Added plugin 'org.apache.kafka.connect.tools.VerifiableSourceConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,153] INFO Added plugin 'org.apache.kafka.connect.tools.MockSinkConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,153] INFO Added plugin 'org.apache.kafka.connect.mirror.MirrorHeartbeatConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,153] INFO Added plugin 'org.apache.kafka.connect.tools.MockConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,153] INFO Added plugin 'org.apache.kafka.connect.converters.FloatConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,153] INFO Added plugin 'org.apache.kafka.connect.converters.DoubleConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,153] INFO Added plugin 'io.confluent.connect.avro.AvroConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,154] INFO Added plugin 'org.apache.kafka.connect.converters.ByteArrayConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,154] INFO Added plugin 'org.apache.kafka.connect.converters.LongConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,154] INFO Added plugin 'org.apache.kafka.connect.converters.IntegerConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,154] INFO Added plugin 'io.confluent.connect.json.JsonSchemaConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,154] INFO Added plugin 'io.confluent.connect.protobuf.ProtobufConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,154] INFO Added plugin 'org.apache.kafka.connect.json.JsonConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,154] INFO Added plugin 'org.apache.kafka.connect.storage.StringConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,154] INFO Added plugin 'org.apache.kafka.connect.converters.ShortConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,154] INFO Added plugin 'org.apache.kafka.connect.storage.SimpleHeaderConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,154] INFO Added plugin 'org.apache.kafka.connect.transforms.ReplaceField$Value' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,154] INFO Added plugin 'org.apache.kafka.connect.transforms.SetSchemaMetadata$Value' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,155] INFO Added plugin 'org.apache.kafka.connect.transforms.ReplaceField$Key' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,155] INFO Added plugin 'org.apache.kafka.connect.transforms.InsertField$Value' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,155] INFO Added plugin 'org.apache.kafka.connect.transforms.TimestampConverter$Key' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,155] INFO Added plugin 'org.apache.kafka.connect.transforms.MaskField$Value' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,155] INFO Added plugin 'org.apache.kafka.connect.transforms.TimestampRouter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,155] INFO Added plugin 'org.apache.kafka.connect.transforms.RegexRouter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,155] INFO Added plugin 'org.apache.kafka.connect.transforms.HoistField$Value' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,155] INFO Added plugin 'org.apache.kafka.connect.transforms.ValueToKey' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,155] INFO Added plugin 'org.apache.kafka.connect.transforms.MaskField$Key' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,156] INFO Added plugin 'org.apache.kafka.connect.transforms.Cast$Key' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,156] INFO Added plugin 'org.apache.kafka.connect.transforms.Cast$Value' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,156] INFO Added plugin 'org.apache.kafka.connect.transforms.ExtractField$Key' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,156] INFO Added plugin 'org.apache.kafka.connect.transforms.Flatten$Value' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,156] INFO Added plugin 'org.apache.kafka.connect.transforms.InsertField$Key' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,156] INFO Added plugin 'org.apache.kafka.connect.transforms.Flatten$Key' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,156] INFO Added plugin 'org.apache.kafka.connect.transforms.SetSchemaMetadata$Key' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,156] INFO Added plugin 'org.apache.kafka.connect.transforms.ExtractField$Value' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,157] INFO Added plugin 'org.apache.kafka.connect.transforms.TimestampConverter$Value' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,157] INFO Added plugin 'org.apache.kafka.connect.transforms.HoistField$Key' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,157] INFO Added plugin 'org.apache.kafka.common.config.provider.FileConfigProvider' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,157] INFO Added plugin 'org.apache.kafka.connect.rest.basic.auth.extension.BasicAuthSecurityRestExtension' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,159] INFO Added aliases 'JdbcSinkConnector' and 'JdbcSink' to plugin 'io.confluent.connect.jdbc.JdbcSinkConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,160] INFO Added aliases 'JdbcSourceConnector' and 'JdbcSource' to plugin 'io.confluent.connect.jdbc.JdbcSourceConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,160] INFO Added aliases 'FileStreamSinkConnector' and 'FileStreamSink' to plugin 'org.apache.kafka.connect.file.FileStreamSinkConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,160] INFO Added aliases 'FileStreamSourceConnector' and 'FileStreamSource' to plugin 'org.apache.kafka.connect.file.FileStreamSourceConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,160] INFO Added aliases 'MirrorCheckpointConnector' and 'MirrorCheckpoint' to plugin 'org.apache.kafka.connect.mirror.MirrorCheckpointConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,160] INFO Added aliases 'MirrorHeartbeatConnector' and 'MirrorHeartbeat' to plugin 'org.apache.kafka.connect.mirror.MirrorHeartbeatConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,160] INFO Added aliases 'MirrorSourceConnector' and 'MirrorSource' to plugin 'org.apache.kafka.connect.mirror.MirrorSourceConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,161] INFO Added aliases 'MockConnector' and 'Mock' to plugin 'org.apache.kafka.connect.tools.MockConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,161] INFO Added aliases 'MockSinkConnector' and 'MockSink' to plugin 'org.apache.kafka.connect.tools.MockSinkConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,161] INFO Added aliases 'MockSourceConnector' and 'MockSource' to plugin 'org.apache.kafka.connect.tools.MockSourceConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,161] INFO Added aliases 'SchemaSourceConnector' and 'SchemaSource' to plugin 'org.apache.kafka.connect.tools.SchemaSourceConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,161] INFO Added aliases 'VerifiableSinkConnector' and 'VerifiableSink' to plugin 'org.apache.kafka.connect.tools.VerifiableSinkConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,161] INFO Added aliases 'VerifiableSourceConnector' and 'VerifiableSource' to plugin 'org.apache.kafka.connect.tools.VerifiableSourceConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,162] INFO Added aliases 'AvroConverter' and 'Avro' to plugin 'io.confluent.connect.avro.AvroConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,162] INFO Added aliases 'JsonSchemaConverter' and 'JsonSchema' to plugin 'io.confluent.connect.json.JsonSchemaConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,162] INFO Added aliases 'ProtobufConverter' and 'Protobuf' to plugin 'io.confluent.connect.protobuf.ProtobufConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,162] INFO Added aliases 'ByteArrayConverter' and 'ByteArray' to plugin 'org.apache.kafka.connect.converters.ByteArrayConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,162] INFO Added aliases 'DoubleConverter' and 'Double' to plugin 'org.apache.kafka.connect.converters.DoubleConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,162] INFO Added aliases 'FloatConverter' and 'Float' to plugin 'org.apache.kafka.connect.converters.FloatConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,163] INFO Added aliases 'IntegerConverter' and 'Integer' to plugin 'org.apache.kafka.connect.converters.IntegerConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,163] INFO Added aliases 'LongConverter' and 'Long' to plugin 'org.apache.kafka.connect.converters.LongConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,163] INFO Added aliases 'ShortConverter' and 'Short' to plugin 'org.apache.kafka.connect.converters.ShortConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,163] INFO Added aliases 'JsonConverter' and 'Json' to plugin 'org.apache.kafka.connect.json.JsonConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,163] INFO Added aliases 'StringConverter' and 'String' to plugin 'org.apache.kafka.connect.storage.StringConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,163] INFO Added aliases 'ByteArrayConverter' and 'ByteArray' to plugin 'org.apache.kafka.connect.converters.ByteArrayConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,163] INFO Added aliases 'DoubleConverter' and 'Double' to plugin 'org.apache.kafka.connect.converters.DoubleConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,164] INFO Added aliases 'FloatConverter' and 'Float' to plugin 'org.apache.kafka.connect.converters.FloatConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,164] INFO Added aliases 'IntegerConverter' and 'Integer' to plugin 'org.apache.kafka.connect.converters.IntegerConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,164] INFO Added aliases 'LongConverter' and 'Long' to plugin 'org.apache.kafka.connect.converters.LongConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,164] INFO Added aliases 'ShortConverter' and 'Short' to plugin 'org.apache.kafka.connect.converters.ShortConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,164] INFO Added aliases 'JsonConverter' and 'Json' to plugin 'org.apache.kafka.connect.json.JsonConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,164] INFO Added alias 'SimpleHeaderConverter' to plugin 'org.apache.kafka.connect.storage.SimpleHeaderConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,164] INFO Added aliases 'StringConverter' and 'String' to plugin 'org.apache.kafka.connect.storage.StringConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,165] INFO Added alias 'ChangeTopicCase' to plugin 'com.github.jcustenborder.kafka.connect.transform.common.ChangeTopicCase' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,166] INFO Added alias 'HeaderToField' to plugin 'com.github.jcustenborder.kafka.connect.transform.common.HeaderToField' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,167] INFO Added alias 'SetMaximumPrecision' to plugin 'com.github.jcustenborder.kafka.connect.transform.common.SetMaximumPrecision' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,167] INFO Added alias 'TimestampNow' to plugin 'com.github.jcustenborder.kafka.connect.transform.common.TimestampNow' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,168] INFO Added alias 'RegexRouter' to plugin 'org.apache.kafka.connect.transforms.RegexRouter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,169] INFO Added alias 'TimestampRouter' to plugin 'org.apache.kafka.connect.transforms.TimestampRouter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,169] INFO Added alias 'ValueToKey' to plugin 'org.apache.kafka.connect.transforms.ValueToKey' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,169] INFO Added alias 'BasicAuthSecurityRestExtension' to plugin 'org.apache.kafka.connect.rest.basic.auth.extension.BasicAuthSecurityRestExtension' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,169] INFO Added aliases 'AllConnectorClientConfigOverridePolicy' and 'All' to plugin 'org.apache.kafka.connect.connector.policy.AllConnectorClientConfigOverridePolicy' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,169] INFO Added aliases 'NoneConnectorClientConfigOverridePolicy' and 'None' to plugin 'org.apache.kafka.connect.connector.policy.NoneConnectorClientConfigOverridePolicy' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,170] INFO Added aliases 'PrincipalConnectorClientConfigOverridePolicy' and 'Principal' to plugin 'org.apache.kafka.connect.connector.policy.PrincipalConnectorClientConfigOverridePolicy' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
@volodymyr-kushnir Did you cut that log together or is that one continuous log entry? I'm seeing my plugin loaded with the Apache Kafka Plugins as well. What do you have as the setting for plugin.path
? I ran your docker-compose and your config. It worked and loaded the transform. What ENV settings are you launching the container with?
That was continuous output, preceeded by this —
===> Launching ...
===> Launching kafka-connect ...
[2020-05-25 16:48:14,718] INFO WorkerInfo values:
jvm.args = -Xms256M, -Xmx2G, -XX:+UseG1GC, -XX:MaxGCPauseMillis=20, -XX:InitiatingHeapOccupancyPercent=35, -XX:+ExplicitGCInvokesConcurrent, -XX:MaxInlineLevel=15, -Djava.awt.headless=true, -Dcom.sun.management.jmxremote=true, -Dcom.sun.management.jmxremote.authenticate=false, -Dcom.sun.management.jmxremote.ssl=false, -Dkafka.logs.dir=/var/log/kafka, -Dlog4j.configuration=file:/etc/kafka/connect-log4j.properties
jvm.spec = Azul Systems, Inc., OpenJDK 64-Bit Server VM, 1.8.0_212, 25.212-b04
jvm.classpath = /etc/kafka-connect/jars/*:/usr/share/java/kafka/jersey-container-servlet-core-2.28.jar:/usr/share/java/kafka/jetty-util-9.4.24.v20191120.jar:/usr/share/java/kafka/jakarta.annotation-api-1.3.4.jar:/usr/share/java/kafka/scala-library-2.12.10.jar:/usr/share/java/kafka/commons-compress-1.19.jar:/usr/share/java/kafka/commons-lang3-3.8.1.jar:/usr/share/java/kafka/scala-java8-compat_2.12-0.9.0.jar:/usr/share/java/kafka/aopalliance-repackaged-2.5.0.jar:/usr/share/java/kafka/connect-basic-auth-extension-5.5.0-ccs.jar:/usr/share/java/kafka/kafka-streams-examples-5.5.0-ccs.jar:/usr/share/java/kafka/kafka.jar:/usr/share/java/kafka/jakarta.ws.rs-api-2.1.5.jar:/usr/share/java/kafka/jetty-io-9.4.24.v20191120.jar:/usr/share/java/kafka/metrics-core-2.2.0.jar:/usr/share/java/kafka/avro-1.9.2.jar:/usr/share/java/kafka/netty-handler-4.1.45.Final.jar:/usr/share/java/kafka/commons-logging-1.2.jar:/usr/share/java/kafka/jetty-servlets-9.4.24.v20191120.jar:/usr/share/java/kafka/httpmime-4.5.11.jar:/usr/share/java/kafka/connect-mirror-client-5.5.0-ccs.jar:/usr/share/java/kafka/kafka_2.12-5.5.0-ccs-javadoc.jar:/usr/share/java/kafka/jackson-databind-2.10.2.jar:/usr/share/java/kafka/scala-logging_2.12-3.9.2.jar:/usr/share/java/kafka/plexus-utils-3.2.1.jar:/usr/share/java/kafka/hk2-locator-2.5.0.jar:/usr/share/java/kafka/connect-json-5.5.0-ccs.jar:/usr/share/java/kafka/jersey-client-2.28.jar:/usr/share/java/kafka/reflections-0.9.12.jar:/usr/share/java/kafka/zstd-jni-1.4.4-7.jar:/usr/share/java/kafka/connect-mirror-5.5.0-ccs.jar:/usr/share/java/kafka/kafka-log4j-appender-5.5.0-ccs.jar:/usr/share/java/kafka/connect-file-5.5.0-ccs.jar:/usr/share/java/kafka/netty-common-4.1.45.Final.jar:/usr/share/java/kafka/audience-annotations-0.5.0.jar:/usr/share/java/kafka/log4j-1.2.17.jar:/usr/share/java/kafka/lz4-java-1.7.1.jar:/usr/share/java/kafka/kafka-streams-scala_2.12-5.5.0-ccs.jar:/usr/share/java/kafka/jackson-module-paranamer-2.10.2.jar:/usr/share/java/kafka/support-metrics-client-5.5.0-ccs.jar:/usr/share/java/kafka/jackson-dataformat-csv-2.10.2.jar:/usr/share/java/kafka/jackson-module-jaxb-annotations-2.10.2.jar:/usr/share/java/kafka/argparse4j-0.7.0.jar:/usr/share/java/kafka/netty-codec-4.1.45.Final.jar:/usr/share/java/kafka/zookeeper-jute-3.5.7.jar:/usr/share/java/kafka/netty-transport-native-unix-common-4.1.45.Final.jar:/usr/share/java/kafka/kafka_2.12-5.5.0-ccs-scaladoc.jar:/usr/share/java/kafka/jackson-module-scala_2.12-2.10.2.jar:/usr/share/java/kafka/kafka_2.12-5.5.0-ccs-test-sources.jar:/usr/share/java/kafka/jersey-container-servlet-2.28.jar:/usr/share/java/kafka/commons-codec-1.11.jar:/usr/share/java/kafka/kafka_2.12-5.5.0-ccs-sources.jar:/usr/share/java/kafka/slf4j-api-1.7.30.jar:/usr/share/java/kafka/netty-resolver-4.1.45.Final.jar:/usr/share/java/kafka/jakarta.activation-api-1.2.1.jar:/usr/share/java/kafka/maven-artifact-3.6.3.jar:/usr/share/java/kafka/jackson-datatype-jdk8-2.10.2.jar:/usr/share/java/kafka/httpcore-4.4.13.jar:/usr/share/java/kafka/javax.ws.rs-api-2.1.1.jar:/usr/share/java/kafka/hk2-api-2.5.0.jar:/usr/share/java/kafka/jakarta.inject-2.5.0.jar:/usr/share/java/kafka/osgi-resource-locator-1.0.1.jar:/usr/share/java/kafka/activation-1.1.1.jar:/usr/share/java/kafka/zookeeper-3.5.7.jar:/usr/share/java/kafka/scala-reflect-2.12.10.jar:/usr/share/java/kafka/rocksdbjni-5.18.3.jar:/usr/share/java/kafka/jaxb-api-2.3.0.jar:/usr/share/java/kafka/commons-cli-1.4.jar:/usr/share/java/kafka/connect-transforms-5.5.0-ccs.jar:/usr/share/java/kafka/kafka_2.12-5.5.0-ccs.jar:/usr/share/java/kafka/jersey-common-2.28.jar:/usr/share/java/kafka/jersey-hk2-2.28.jar:/usr/share/java/kafka/jetty-continuation-9.4.24.v20191120.jar:/usr/share/java/kafka/jackson-core-2.10.2.jar:/usr/share/java/kafka/javassist-3.22.0-CR2.jar:/usr/share/java/kafka/javax.servlet-api-3.1.0.jar:/usr/share/java/kafka/jetty-http-9.4.24.v20191120.jar:/usr/share/java/kafka/validation-api-2.0.1.Final.jar:/usr/share/java/kafka/support-metrics-common-5.5.0-ccs.jar:/usr/share/java/kafka/jersey-media-jaxb-2.28.jar:/usr/share/java/kafka/kafka-clients-5.5.0-ccs.jar:/usr/share/java/kafka/jackson-jaxrs-json-provider-2.10.2.jar:/usr/share/java/kafka/scala-collection-compat_2.12-2.1.3.jar:/usr/share/java/kafka/kafka-tools-5.5.0-ccs.jar:/usr/share/java/kafka/javassist-3.26.0-GA.jar:/usr/share/java/kafka/jopt-simple-5.0.4.jar:/usr/share/java/kafka/connect-api-5.5.0-ccs.jar:/usr/share/java/kafka/paranamer-2.8.jar:/usr/share/java/kafka/snappy-java-1.1.7.3.jar:/usr/share/java/kafka/jetty-security-9.4.24.v20191120.jar:/usr/share/java/kafka/connect-runtime-5.5.0-ccs.jar:/usr/share/java/kafka/jetty-client-9.4.24.v20191120.jar:/usr/share/java/kafka/hk2-utils-2.5.0.jar:/usr/share/java/kafka/jakarta.xml.bind-api-2.3.2.jar:/usr/share/java/kafka/netty-buffer-4.1.45.Final.jar:/usr/share/java/kafka/kafka-streams-5.5.0-ccs.jar:/usr/share/java/kafka/jersey-server-2.28.jar:/usr/share/java/kafka/jetty-server-9.4.24.v20191120.jar:/usr/share/java/kafka/kafka_2.12-5.5.0-ccs-test.jar:/usr/share/java/kafka/jackson-annotations-2.10.2.jar:/usr/share/java/kafka/netty-transport-native-epoll-4.1.45.Final.jar:/usr/share/java/kafka/kafka-streams-test-utils-5.5.0-ccs.jar:/usr/share/java/kafka/httpclient-4.5.11.jar:/usr/share/java/kafka/jetty-servlet-9.4.24.v20191120.jar:/usr/share/java/kafka/slf4j-log4j12-1.7.30.jar:/usr/share/java/kafka/netty-transport-4.1.45.Final.jar:/usr/share/java/kafka/jackson-jaxrs-base-2.10.2.jar:/usr/share/java/kafka/kafka-connect-jdbc-6.0.0-SNAPSHOT.jar:/usr/share/java/kafka/slf4j-api-1.7.26.jar:/usr/share/java/kafka/sqlite-jdbc-3.25.2.jar:/usr/share/java/kafka/postgresql-42.2.10.jar:/usr/share/java/kafka/jtds-1.3.1.jar:/usr/share/java/kafka/common-utils-6.0.0-SNAPSHOT.jar:/usr/share/java/kafka/confluent-metrics-5.5.0-ce.jar:/usr/share/java/confluent-common/common-metrics-5.5.0.jar:/usr/share/java/confluent-common/slf4j-api-1.7.26.jar:/usr/share/java/confluent-common/build-tools-5.5.0.jar:/usr/share/java/confluent-common/common-utils-5.5.0.jar:/usr/share/java/confluent-common/common-config-5.5.0.jar:/usr/share/java/kafka-serde-tools/kafka-avro-serializer-5.5.0.jar:/usr/share/java/kafka-serde-tools/wire-runtime-3.1.0.jar:/usr/share/java/kafka-serde-tools/jackson-datatype-joda-2.10.2.jar:/usr/share/java/kafka-serde-tools/kafka-connect-avro-data-5.5.0.jar:/usr/share/java/kafka-serde-tools/scala-library-2.12.10.jar:/usr/share/java/kafka-serde-tools/commons-compress-1.19.jar:/usr/share/java/kafka-serde-tools/commons-lang3-3.8.1.jar:/usr/share/java/kafka-serde-tools/classgraph-4.8.21.jar:/usr/share/java/kafka-serde-tools/jakarta.inject-2.6.1.jar:/usr/share/java/kafka-serde-tools/avro-1.9.2.jar:/usr/share/java/kafka-serde-tools/jersey-client-2.30.jar:/usr/share/java/kafka-serde-tools/jersey-common-2.30.jar:/usr/share/java/kafka-serde-tools/kafka-streams-avro-serde-5.5.0.jar:/usr/share/java/kafka-serde-tools/kotlin-scripting-compiler-embeddable-1.3.50.jar:/usr/share/java/kafka-serde-tools/commons-logging-1.2.jar:/usr/share/java/kafka-serde-tools/failureaccess-1.0.1.jar:/usr/share/java/kafka-serde-tools/swagger-core-1.5.3.jar:/usr/share/java/kafka-serde-tools/jboss-logging-3.3.2.Final.jar:/usr/share/java/kafka-serde-tools/jackson-databind-2.10.2.jar:/usr/share/java/kafka-serde-tools/mbknor-jackson-jsonschema_2.12-1.0.36.jar:/usr/share/java/kafka-serde-tools/protobuf-java-util-3.11.4.jar:/usr/share/java/kafka-serde-tools/kotlin-stdlib-jdk8-1.3.61.jar:/usr/share/java/kafka-serde-tools/j2objc-annotations-1.3.jar:/usr/share/java/kafka-serde-tools/kafka-connect-protobuf-converter-5.5.0.jar:/usr/share/java/kafka-serde-tools/protobuf-java-3.11.4.jar:/usr/share/java/kafka-serde-tools/kafka-protobuf-provider-5.5.0.jar:/usr/share/java/kafka-serde-tools/checker-compat-qual-2.5.5.jar:/usr/share/java/kafka-serde-tools/commons-digester-1.8.1.jar:/usr/share/java/kafka-serde-tools/commons-collections-3.2.2.jar:/usr/share/java/kafka-serde-tools/json-20190722.jar:/usr/share/java/kafka-serde-tools/snakeyaml-1.24.jar:/usr/share/java/kafka-serde-tools/kafka-streams-protobuf-serde-5.5.0.jar:/usr/share/java/kafka-serde-tools/osgi-resource-locator-1.0.3.jar:/usr/share/java/kafka-serde-tools/commons-validator-1.6.jar:/usr/sh
are/java/kafka-serde-tools/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:/usr/share/java/kafka-serde-tools/error_prone_annotations-2.3.4.jar:/usr/share/java/kafka-serde-tools/kafka-connect-json-schema-converter-5.5.0.jar:/usr/share/java/kafka-serde-tools/kotlin-stdlib-jdk7-1.3.61.jar:/usr/share/java/kafka-serde-tools/jersey-media-jaxb-2.30.jar:/usr/share/java/kafka-serde-tools/kotlinx-coroutines-core-1.1.1.jar:/usr/share/java/kafka-serde-tools/jakarta.el-3.0.2.jar:/usr/share/java/kafka-serde-tools/jackson-datatype-jsr310-2.10.2.jar:/usr/share/java/kafka-serde-tools/kotlin-stdlib-common-1.3.61.jar:/usr/share/java/kafka-serde-tools/jakarta.el-api-3.0.3.jar:/usr/share/java/kafka-serde-tools/kafka-json-schema-serializer-5.5.0.jar:/usr/share/java/kafka-serde-tools/kotlin-reflect-1.3.50.jar:/usr/share/java/kafka-serde-tools/jsr305-3.0.2.jar:/usr/share/java/kafka-serde-tools/joda-time-2.10.2.jar:/usr/share/java/kafka-serde-tools/jackson-datatype-jdk8-2.10.2.jar:/usr/share/java/kafka-serde-tools/jakarta.annotation-api-1.3.5.jar:/usr/share/java/kafka-serde-tools/kafka-schema-registry-client-5.5.0.jar:/usr/share/java/kafka-serde-tools/kafka-schema-serializer-5.5.0.jar:/usr/share/java/kafka-serde-tools/handy-uri-templates-2.1.8.jar:/usr/share/java/kafka-serde-tools/guava-28.1-android.jar:/usr/share/java/kafka-serde-tools/kafka-streams-json-schema-serde-5.5.0.jar:/usr/share/java/kafka-serde-tools/hibernate-validator-6.0.17.Final.jar:/usr/share/java/kafka-serde-tools/rocksdbjni-5.18.3.jar:/usr/share/java/kafka-serde-tools/re2j-1.3.jar:/usr/share/java/kafka-serde-tools/gson-2.8.5.jar:/usr/share/java/kafka-serde-tools/jersey-bean-validation-2.30.jar:/usr/share/java/kafka-serde-tools/kafka-protobuf-serializer-5.5.0.jar:/usr/share/java/kafka-serde-tools/jackson-core-2.10.2.jar:/usr/share/java/kafka-serde-tools/swagger-models-1.5.3.jar:/usr/share/java/kafka-serde-tools/kotlinx-coroutines-core-common-1.1.1.jar:/usr/share/java/kafka-serde-tools/kotlin-scripting-common-1.3.50.jar:/usr/share/java/kafka-serde-tools/kafka-connect-avro-converter-5.5.0.jar:/usr/share/java/kafka-serde-tools/classmate-1.3.4.jar:/usr/share/java/kafka-serde-tools/validation-api-2.0.1.Final.jar:/usr/share/java/kafka-serde-tools/annotations-13.0.jar:/usr/share/java/kafka-serde-tools/jersey-server-2.30.jar:/usr/share/java/kafka-serde-tools/jackson-module-parameter-names-2.10.2.jar:/usr/share/java/kafka-serde-tools/animal-sniffer-annotations-1.18.jar:/usr/share/java/kafka-serde-tools/kafka-json-schema-provider-5.5.0.jar:/usr/share/java/kafka-serde-tools/jakarta.validation-api-2.0.2.jar:/usr/share/java/kafka-serde-tools/kotlin-stdlib-1.3.61.jar:/usr/share/java/kafka-serde-tools/jackson-dataformat-yaml-2.10.2.jar:/usr/share/java/kafka-serde-tools/wire-schema-3.1.0.jar:/usr/share/java/kafka-serde-tools/okio-2.4.3.jar:/usr/share/java/kafka-serde-tools/swagger-annotations-1.5.22.jar:/usr/share/java/kafka-serde-tools/kotlin-scripting-compiler-impl-embeddable-1.3.50.jar:/usr/share/java/kafka-serde-tools/kotlin-scripting-jvm-1.3.50.jar:/usr/share/java/kafka-serde-tools/jackson-datatype-guava-2.10.2.jar:/usr/share/java/kafka-serde-tools/jakarta.ws.rs-api-2.1.6.jar:/usr/share/java/kafka-serde-tools/kafka-streams-5.5.0-ccs.jar:/usr/share/java/kafka-serde-tools/kafka-json-serializer-5.5.0.jar:/usr/share/java/kafka-serde-tools/jackson-annotations-2.10.2.jar:/usr/share/java/kafka-serde-tools/org.everit.json.schema-1.12.1.jar:/usr/share/java/kafka-serde-tools/kotlin-script-runtime-1.3.50.jar:/usr/share/java/monitoring-interceptors/monitoring-interceptors-5.5.0.jar:/usr/bin/../share/java/kafka/jersey-container-servlet-core-2.28.jar:/usr/bin/../share/java/kafka/jetty-util-9.4.24.v20191120.jar:/usr/bin/../share/java/kafka/jakarta.annotation-api-1.3.4.jar:/usr/bin/../share/java/kafka/scala-library-2.12.10.jar:/usr/bin/../share/java/kafka/commons-compress-1.19.jar:/usr/bin/../share/java/kafka/commons-lang3-3.8.1.jar:/usr/bin/../share/java/kafka/scala-java8-compat_2.12-0.9.0.jar:/usr/bin/../share/java/kafka/aopalliance-repackaged-2.5.0.jar:/usr/bin/../share/java/kafka/connect-basic-auth-extension-5.5.0-ccs.jar:/usr/bin/../share/java/kafka/kafka-streams-examples-5.5.0-ccs.jar:/usr/bin/../share/java/kafka/kafka.jar:/usr/bin/../share/java/kafka/jakarta.ws.rs-api-2.1.5.jar:/usr/bin/../share/java/kafka/jetty-io-9.4.24.v20191120.jar:/usr/bin/../share/java/kafka/metrics-core-2.2.0.jar:/usr/bin/../share/java/kafka/avro-1.9.2.jar:/usr/bin/../share/java/kafka/netty-handler-4.1.45.Final.jar:/usr/bin/../share/java/kafka/commons-logging-1.2.jar:/usr/bin/../share/java/kafka/jetty-servlets-9.4.24.v20191120.jar:/usr/bin/../share/java/kafka/httpmime-4.5.11.jar:/usr/bin/../share/java/kafka/connect-mirror-client-5.5.0-ccs.jar:/usr/bin/../share/java/kafka/kafka_2.12-5.5.0-ccs-javadoc.jar:/usr/bin/../share/java/kafka/jackson-databind-2.10.2.jar:/usr/bin/../share/java/kafka/scala-logging_2.12-3.9.2.jar:/usr/bin/../share/java/kafka/plexus-utils-3.2.1.jar:/usr/bin/../share/java/kafka/hk2-locator-2.5.0.jar:/usr/bin/../share/java/kafka/connect-json-5.5.0-ccs.jar:/usr/bin/../share/java/kafka/jersey-client-2.28.jar:/usr/bin/../share/java/kafka/reflections-0.9.12.jar:/usr/bin/../share/java/kafka/zstd-jni-1.4.4-7.jar:/usr/bin/../share/java/kafka/connect-mirror-5.5.0-ccs.jar:/usr/bin/../share/java/kafka/kafka-log4j-appender-5.5.0-ccs.jar:/usr/bin/../share/java/kafka/connect-file-5.5.0-ccs.jar:/usr/bin/../share/java/kafka/netty-common-4.1.45.Final.jar:/usr/bin/../share/java/kafka/audience-annotations-0.5.0.jar:/usr/bin/../share/java/kafka/log4j-1.2.17.jar:/usr/bin/../share/java/kafka/lz4-java-1.7.1.jar:/usr/bin/../share/java/kafka/kafka-streams-scala_2.12-5.5.0-ccs.jar:/usr/bin/../share/java/kafka/jackson-module-paranamer-2.10.2.jar:/usr/bin/../share/java/kafka/support-metrics-client-5.5.0-ccs.jar:/usr/bin/../share/java/kafka/jackson-dataformat-csv-2.10.2.jar:/usr/bin/../share/java/kafka/jackson-module-jaxb-annotations-2.10.2.jar:/usr/bin/../share/java/kafka/argparse4j-0.7.0.jar:/usr/bin/../share/java/kafka/netty-codec-4.1.45.Final.jar:/usr/bin/../share/java/kafka/zookeeper-jute-3.5.7.jar:/usr/bin/../share/java/kafka/netty-transport-native-unix-common-4.1.45.Final.jar:/usr/bin/../share/java/kafka/kafka_2.12-5.5.0-ccs-scaladoc.jar:/usr/bin/../share/java/kafka/jackson-module-scala_2.12-2.10.2.jar:/usr/bin/../share/java/kafka/kafka_2.12-5.5.0-ccs-test-sources.jar:/usr/bin/../share/java/kafka/jersey-container-servlet-2.28.jar:/usr/bin/../share/java/kafka/commons-codec-1.11.jar:/usr/bin/../share/java/kafka/kafka_2.12-5.5.0-ccs-sources.jar:/usr/bin/../share/java/kafka/slf4j-api-1.7.30.jar:/usr/bin/../share/java/kafka/netty-resolver-4.1.45.Final.jar:/usr/bin/../share/java/kafka/jakarta.activation-api-1.2.1.jar:/usr/bin/../share/java/kafka/maven-artifact-3.6.3.jar:/usr/bin/../share/java/kafka/jackson-datatype-jdk8-2.10.2.jar:/usr/bin/../share/java/kafka/httpcore-4.4.13.jar:/usr/bin/../share/java/kafka/javax.ws.rs-api-2.1.1.jar:/usr/bin/../share/java/kafka/hk2-api-2.5.0.jar:/usr/bin/../share/java/kafka/jakarta.inject-2.5.0.jar:/usr/bin/../share/java/kafka/osgi-resource-locator-1.0.1.jar:/usr/bin/../share/java/kafka/activation-1.1.1.jar:/usr/bin/../share/java/kafka/zookeeper-3.5.7.jar:/usr/bin/../share/java/kafka/scala-reflect-2.12.10.jar:/usr/bin/../share/java/kafka/rocksdbjni-5.18.3.jar:/usr/bin/../share/java/kafka/jaxb-api-2.3.0.jar:/usr/bin/../share/java/kafka/commons-cli-1.4.jar:/usr/bin/../share/java/kafka/connect-transforms-5.5.0-ccs.jar:/usr/bin/../share/java/kafka/kafka_2.12-5.5.0-ccs.jar:/usr/bin/../share/java/kafka/jersey-common-2.28.jar:/usr/bin/../share/java/kafka/jersey-hk2-2.28.jar:/usr/bin/../share/java/kafka/jetty-continuation-9.4.24.v20191120.jar:/usr/bin/../share/java/kafka/jackson-core-2.10.2.jar:/usr/bin/../share/java/kafka/javassist-3.22.0-CR2.jar:/usr/bin/../share/java/kafka/javax.servlet-api-3.1.0.jar:/usr/bin/../share/java/kafka/jetty-http-9.4.24.v20191120.jar:/usr/bin/../share/java/kafka/validation-api-2.0.1.Final.jar:/usr/bin/../share/java/kafka/support-metrics-common-5.5.0-ccs.jar:/usr/bin/../share/java/kafka/jersey-media-jaxb-2.28.jar:/usr/bin/../share/java/kafka/kafka-clients-5.5.0-ccs.jar:/usr/bin/../
share/java/kaf
ka/jackson-jaxrs-json-provider-2.10.2.jar:/usr/bin/../share/java/kafka/scala-collection-compat_2.12-2.1.3.jar:/usr/bin/../share/java/kafka/kafka-tools-5.5.0-ccs.jar:/usr/bin/../share/java/kafka/javassist-3.26.0-GA.jar:/usr/bin/../share/java/kafka/jopt-simple-5.0.4.jar:/usr/bin/../share/java/kafka/connect-api-5.5.0-ccs.jar:/usr/bin/../share/java/kafka/paranamer-2.8.jar:/usr/bin/../share/java/kafka/snappy-java-1.1.7.3.jar:/usr/bin/../share/java/kafka/jetty-security-9.4.24.v20191120.jar:/usr/bin/../share/java/kafka/connect-runtime-5.5.0-ccs.jar:/usr/bin/../share/java/kafka/jetty-client-9.4.24.v20191120.jar:/usr/bin/../share/java/kafka/hk2-utils-2.5.0.jar:/usr/bin/../share/java/kafka/jakarta.xml.bind-api-2.3.2.jar:/usr/bin/../share/java/kafka/netty-buffer-4.1.45.Final.jar:/usr/bin/../share/java/kafka/kafka-streams-5.5.0-ccs.jar:/usr/bin/../share/java/kafka/jersey-server-2.28.jar:/usr/bin/../share/java/kafka/jetty-server-9.4.24.v20191120.jar:/usr/bin/../share/java/kafka/kafka_2.12-5.5.0-ccs-test.jar:/usr/bin/../share/java/kafka/jackson-annotations-2.10.2.jar:/usr/bin/../share/java/kafka/netty-transport-native-epoll-4.1.45.Final.jar:/usr/bin/../share/java/kafka/kafka-streams-test-utils-5.5.0-ccs.jar:/usr/bin/../share/java/kafka/httpclient-4.5.11.jar:/usr/bin/../share/java/kafka/jetty-servlet-9.4.24.v20191120.jar:/usr/bin/../share/java/kafka/slf4j-log4j12-1.7.30.jar:/usr/bin/../share/java/kafka/netty-transport-4.1.45.Final.jar:/usr/bin/../share/java/kafka/jackson-jaxrs-base-2.10.2.jar:/usr/bin/../share/java/kafka/kafka-connect-jdbc-6.0.0-SNAPSHOT.jar:/usr/bin/../share/java/kafka/slf4j-api-1.7.26.jar:/usr/bin/../share/java/kafka/sqlite-jdbc-3.25.2.jar:/usr/bin/../share/java/kafka/postgresql-42.2.10.jar:/usr/bin/../share/java/kafka/jtds-1.3.1.jar:/usr/bin/../share/java/kafka/common-utils-6.0.0-SNAPSHOT.jar:/usr/bin/../share/java/kafka/confluent-metrics-5.5.0-ce.jar:/usr/bin/../support-metrics-client/build/dependant-libs-2.12/*:/usr/bin/../support-metrics-client/build/libs/*:/usr/share/java/support-metrics-client/*
os.spec = Linux, amd64, 4.19.76-linuxkit
os.vcpus = 6
(org.apache.kafka.connect.runtime.WorkerInfo)
And I run Docker container with these —
docker run -d \
--name=<name> \
--publish=80:80 \
-e CONNECT_BOOTSTRAP_SERVERS="<hostname>:<port>" \
-e CONNECT_REST_PORT="80" \
-e CONNECT_GROUP_ID="hive-service-connect" \
-e CONNECT_CONFIG_STORAGE_TOPIC="<name>" \
-e CONNECT_OFFSET_STORAGE_TOPIC="<name>" \
-e CONNECT_STATUS_STORAGE_TOPIC="<name>" \
-e CONNECT_KEY_CONVERTER="org.apache.kafka.connect.json.JsonConverter" \
-e CONNECT_VALUE_CONVERTER="org.apache.kafka.connect.json.JsonConverter" \
-e CONNECT_INTERNAL_KEY_CONVERTER="org.apache.kafka.connect.json.JsonConverter" \
-e CONNECT_INTERNAL_VALUE_CONVERTER="org.apache.kafka.connect.json.JsonConverter" \
-e CONNECT_REST_ADVERTISED_HOST_NAME="localhost" \
-e CONNECT_SSL_ENDPOINT_IDENTIFICATION_ALGORITHM="https" \
-e CONNECT_SASL_MECHANISM="PLAIN" \
-e CONNECT_REQUEST_TIMEOUT_MS="20000" \
-e CONNECT_RETRY_BACKOFF_MS="500" \
-e CONNECT_SASL_JAAS_CONFIG="<jaas_config>" \
-e CONNECT_SECURITY_PROTOCOL="SASL_SSL" \
-e CONNECT_CONSUMER_SSL_ENDPOINT_IDENTIFICATION_ALGORITHM="https" \
-e CONNECT_CONSUMER_SASL_MECHANISM="PLAIN" \
-e CONNECT_CONSUMER_REQUEST_TIMEOUT_MS="20000" \
-e CONNECT_CONSUMER_RETRY_BACKOFF_MS="500" \
-e CONNECT_CONSUMER_SASL_JAAS_CONFIG="<jaas_config>" \
-e CONNECT_CONSUMER_SECURITY_PROTOCOL="SASL_SSL" \
-e CONNECT_PRODUCER_SSL_ENDPOINT_IDENTIFICATION_ALGORITHM="https" \
-e CONNECT_PRODUCER_SASL_MECHANISM="PLAIN" \
-e CONNECT_PRODUCER_REQUEST_TIMEOUT_MS="20000" \
-e CONNECT_PRODUCER_RETRY_BACKOFF_MS="500" \
-e CONNECT_PRODUCER_SASL_JAAS_CONFIG="<jaas_config>" \
-e CONNECT_PRODUCER_SECURITY_PROTOCOL="SASL_SSL" \
-e CONNECT_PLUGIN_PATH=/usr/share/confluent-hub-components \
name:0.1.0
/usr/share/confluent-hub-components is where /jcustenborder-kafka-connect-transform-common folder lands. lib folder has these —
-rw-r--r-- 1 root root 3.4K May 25 14:19 animal-sniffer-annotations-1.17.jar
-rw-r--r-- 1 root root 189K May 25 14:19 checker-qual-2.5.2.jar
-rw-r--r-- 1 root root 148K May 25 14:19 connect-utils-0.4.162.jar
-rw-r--r-- 1 root root 14K May 25 14:19 error_prone_annotations-2.2.0.jar
-rw-r--r-- 1 root root 4.6K May 25 14:19 failureaccess-1.0.1.jar
-rw-r--r-- 1 root root 1.5M May 25 14:19 freemarker-2.3.25-incubating.jar
-rw-r--r-- 1 root root 2.7M May 25 14:19 guava-27.1-jre.jar
-rw-r--r-- 1 root root 8.6K May 25 14:19 j2objc-annotations-1.1.jar
-rw-r--r-- 1 root root 65K May 25 14:19 jackson-annotations-2.9.0.jar
-rw-r--r-- 1 root root 318K May 25 14:19 jackson-core-2.9.8.jar
-rw-r--r-- 1 root root 1.4M May 25 14:19 jackson-databind-2.9.8.jar
-rw-r--r-- 1 root root 20K May 25 14:19 jsr305-3.0.2.jar
-rw-r--r-- 1 root root 120K May 25 14:19 kafka-connect-transform-common-0.1.0.35.jar
-rw-r--r-- 1 root root 2.2K May 25 14:19 listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar
Seems like this https://github.com/jcustenborder/kafka-connect-transform-xml/issues/18 is related.
Hmmm weird. This is working for me. Can you give it a shot? The only thing I see in your config is CONNECT_REST_ADVERTISED_HOST_NAME
should not be localhost. It should be the hostname of the docker container. I highly doubt that is it though.
version: "3"
services:
kafka:
hostname: kafka
image: confluentinc/cp-enterprise-kafka:5.5.0
depends_on:
- zookeeper
environment:
KAFKA_ZOOKEEPER_CONNECT: "zookeeper:2181"
KAFKA_ADVERTISED_LISTENERS: "PLAINTEXT://:9092"
KAFKA_METRIC_REPORTERS: io.confluent.metrics.reporter.ConfluentMetricsReporter
KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
CONFLUENT_METRICS_REPORTER_BOOTSTRAP_SERVERS: localhost:9092
CONFLUENT_METRICS_REPORTER_ZOOKEEPER_CONNECT: zookeeper:2181
CONFLUENT_METRICS_REPORTER_TOPIC_REPLICAS: 1
ports:
- "9092:9092"
schema-registry:
image: confluentinc/cp-schema-registry:5.5.0
environment:
SCHEMA_REGISTRY_KAFKASTORE_CONNECTION_URL: "zookeeper:2181"
SCHEMA_REGISTRY_HOST_NAME: schema-registry
depends_on:
- zookeeper
- kafka
ports:
- '8081:8081'
zookeeper:
image: confluentinc/cp-zookeeper:5.5.0
environment:
ZOOKEEPER_CLIENT_PORT: "2181"
ZOOKEEPER_TICK_TIME: "2000"
connect:
hostname: connect
build: connect
depends_on:
- schema-registry
- kafka
ports:
- '8083:8083'
environment:
CONNECT_BOOTSTRAP_SERVERS: "kafka:9092"
CONNECT_REST_ADVERTISED_HOST_NAME: connect
CONNECT_PRODUCER_COMPRESSION_TYPE: lz4
CONNECT_GROUP_ID: connect
CONNECT_KEY_CONVERTER: io.confluent.connect.avro.AvroConverter
CONNECT_KEY_CONVERTER_SCHEMA_REGISTRY_URL: http://schema-registry:8081
CONNECT_VALUE_CONVERTER: io.confluent.connect.avro.AvroConverter
CONNECT_VALUE_CONVERTER_SCHEMA_REGISTRY_URL: http://schema-registry:8081
CONNECT_CONFIG_STORAGE_TOPIC: connect_config
CONNECT_CONFIG_STORAGE_REPLICATION_FACTOR: 1
CONNECT_OFFSET_STORAGE_TOPIC: connect_offset
CONNECT_OFFSET_STORAGE_REPLICATION_FACTOR: 1
CONNECT_STATUS_STORAGE_TOPIC: connect_status
CONNECT_STATUS_STORAGE_REPLICATION_FACTOR: 1
CONNECT_INTERNAL_KEY_CONVERTER: org.apache.kafka.connect.json.JsonConverter
CONNECT_INTERNAL_VALUE_CONVERTER: org.apache.kafka.connect.json.JsonConverter
CONNECT_INTERNAL_KEY_CONVERTER_SCHEMAS_ENABLE: 'false'
CONNECT_INTERNAL_VALUE_CONVERTER_SCHEMAS_ENABLE: 'false'
CONNECT_LOG4J_LOGGERS: 'org.reflections=ERROR'
control-center:
image: confluentinc/cp-enterprise-control-center:5.5.0
depends_on:
- zookeeper
- kafka
- connect
ports:
- "9022:9021"
environment:
CONTROL_CENTER_SCHEMA_REGISTRY_URL: "http://schema-registry:8081/"
CONTROL_CENTER_BOOTSTRAP_SERVERS: "kafka:9092"
CONTROL_CENTER_ZOOKEEPER_CONNECT: "zookeeper:2181"
CONTROL_CENTER_CONNECT_CLUSTER: 'connect:8083'
CONTROL_CENTER_REPLICATION_FACTOR: 1
CONTROL_CENTER_KSQL_ENABLE: "false"
connect/Dockerfile
FROM confluentinc/cp-kafka-connect-base:5.5.0
RUN confluent-hub install --no-prompt jcustenborder/kafka-connect-transform-common:latest
Normally I use something like this and set the plugin path in the container. The one difference I'm seeing is that you're only setting /usr/share/confluent-hub-components
and not /usr/share/java,/usr/share/confluent-hub-components
to the plugin path.
FROM confluentinc/cp-kafka-connect:5.5.0
ENV CONNECT_PLUGIN_PATH="/usr/share/java,/usr/share/confluent-hub-components"
RUN confluent-hub install --no-prompt jcustenborder/kafka-connect-transform-common:latest
@jcustenborder thanks so much for your time. I've figured it out. So I followed https://github.com/confluentinc/examples/blob/5.5.0-post/ccloud/connectors/submit_datagen_pageviews_config.sh and the thing is in the shell script it has to be
"transforms.changeCase.type": "com.github.jcustenborder.kafka.connect.transform.common.ChangeCase\$Value",
with a backslash, otherwise it treats $Value
as a variable or something and replaces it with an empty string. And then obviously there's no such plugin on Connect 🤷🏼♂️. It was screaming in my face the whole time, I just couldn't see it.
Mystery solved, case closed. Again, thank you very much!
Don't feel bad. Issues like these are the hardest to figure out.
Had similar issue so in case you're using x.properties the type should be:
transforms.HoistValue.type= org.apache.kafka.connect.transforms.HoistField\$\Value
also to mention if you still get the "Class could not be found" check the class path or load the dependency i classpath
I've built the JAR and put it in the
plugin.path
:When Kafka Connect runs it loads the plugins
But when I try to use a transform I get a
Class… could not be found
error: