jcustenborder / kafka-connect-transform-common

Common Transforms for Kafka Connect.
Apache License 2.0
153 stars 57 forks source link

Class could not be found #37

Closed rmoff closed 4 years ago

rmoff commented 5 years ago

I've built the JAR and put it in the plugin.path:

root@4cc770014999:/# ls -lR /usr/share/java/
/usr/share/java/:
total 80
drwxr-xr-x 2 root root  4096 Dec 15 06:17 confluent-common
drwxr-xr-x 2 root root  4096 Dec 15 06:18 confluent-control-center
drwxr-xr-x 2 root root  4096 Dec 15 06:18 confluent-hub-client
drwxr-xr-x 2 root root  4096 Dec 15 06:18 confluent-rebalancer
-rw-r--r-- 1 root root   957 May  6  2014 java_defaults.mk
drwxr-xr-x 1 root root  4096 Dec 15 06:18 kafka
drwxr-xr-x 2 root root  4096 Dec 15 06:21 kafka-connect-activemq
drwxr-xr-x 2 root root  4096 Dec 15 06:21 kafka-connect-elasticsearch
drwxr-xr-x 2 root root  4096 Dec 15 06:21 kafka-connect-hdfs
drwxr-xr-x 2 root root  4096 Dec 15 06:21 kafka-connect-ibmmq
drwxr-xr-x 2 root root  4096 Dec 15 06:21 kafka-connect-jdbc
drwxr-xr-x 2 root root  4096 Dec 15 06:21 kafka-connect-jms
drwxr-xr-x 2 root root  4096 Dec 15 06:21 kafka-connect-s3
drwxr-xr-x 2 root root 12288 Dec 15 06:21 kafka-connect-storage-common
drwxr-xr-x 9 root root   288 Jan 22 09:23 kafka-connect-transform-common
drwxr-xr-x 2 root root  4096 Dec 15 06:17 kafka-serde-tools
drwxr-xr-x 2 root root  4096 Dec 15 06:18 monitoring-interceptors
drwxr-xr-x 2 root root  4096 Dec 15 06:17 rest-utils
drwxr-xr-x 2 root root  4096 Dec 15 06:17 schema-registry
…
/usr/share/java/kafka-connect-transform-common:
total 5400
-rw-r--r-- 1 root root  133937 Jan 22 09:18 connect-utils-0.3.141.jar
-rw-r--r-- 1 root root 1493680 Jan 22 09:18 freemarker-2.3.25-incubating.jar
-rw-r--r-- 1 root root 2256213 Jan 22 09:19 guava-18.0.jar
-rw-r--r-- 1 root root   55784 Jan 22 09:18 jackson-annotations-2.8.0.jar
-rw-r--r-- 1 root root  281079 Jan 22 09:18 jackson-core-2.8.5.jar
-rw-r--r-- 1 root root 1236315 Jan 22 09:18 jackson-databind-2.8.5.jar
-rw-r--r-- 1 root root   64477 Jan 22 09:22 kafka-connect-transform-common-0.1.0-SNAPSHOT.jar

When Kafka Connect runs it loads the plugins

 [2019-01-22 09:53:36,256] INFO Loading plugin from: /usr/share/java/kafka-connect-transform-common (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
 [2019-01-22 09:53:38,347] INFO Registered loader: PluginClassLoader{pluginLocation=file:/usr/share/java/kafka-connect-transform-common/} (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
 [2019-01-22 09:53:38,348] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.ToLong$Key' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
 [2019-01-22 09:53:38,348] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.BytesToString$Key' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
 [2019-01-22 09:53:38,348] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.ExtractNestedField$Value' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
 [2019-01-22 09:53:38,348] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.TimestampNow' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
 [2019-01-22 09:53:38,364] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.ExtractNestedField$Key' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
 [2019-01-22 09:53:38,365] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.BytesToString$Value' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
 [2019-01-22 09:53:38,365] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.ChangeTopicCase' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
 [2019-01-22 09:53:38,365] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.PatternRename$Key' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
 [2019-01-22 09:53:38,365] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.SetNull$Value' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
 [2019-01-22 09:53:38,366] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.SetNull$Key' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
 [2019-01-22 09:53:38,366] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.PatternRename$Value' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
 [2019-01-22 09:53:38,366] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.PatternFilter$Key' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
 [2019-01-22 09:53:38,366] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.ToJSON$Value' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
 [2019-01-22 09:53:38,367] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.PatternFilter$Value' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
 [2019-01-22 09:53:38,367] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.ToLong$Value' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
 [2019-01-22 09:53:38,367] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.ToJSON$Key' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
 [2019-01-22 09:53:38,367] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.ChangeCase$Key' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
 [2019-01-22 09:53:38,368] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.ExtractTimestamp$Value' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
 [2019-01-22 09:53:38,368] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.ChangeCase$Value' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)

But when I try to use a transform I get a Class… could not be found error:

$ curl -X POST http://localhost:8083/connectors -H "Content-Type: application/json" -d '{
          "name": "file_sink_99",
          "config": {
                  "connector.class": "org.apache.kafka.connect.file.FileStreamSinkConnector",
                  "topics":"test_topic",
                  "value.converter":"org.apache.kafka.connect.json.JsonConverter",
                  "value.converter.schemas.enable": false,
                  "key.converter":"org.apache.kafka.connect.json.JsonConverter",
                  "key.converter.schemas.enable": false,
                  "file":"/data/file_sink_99.txt",
                  "transforms":"tojson",
                  "transforms.tojson.type":"com.github.jcustenborder.kafka.connect.transform.common.ToJSON$Value",
                  "transforms.tojson.schemas.enable":true
                  }
          }'
{"error_code":400,"message":"Connector configuration is invalid and contains the following 2 error(s):\nInvalid value com.github.jcustenborder.kafka.connect.transform.common.ToJSON$Value for configuration transforms.tojson.type: Class com.github.jcustenborder.kafka.connect.transform.common.ToJSON$Value could not be found.\nInvalid value null for configuration transforms.tojson.type: Not a Transformation\nYou can also find the above list of errors at the endpoint `/{connectorType}/config/validate`"}⏎
volodymyr-kushnir commented 4 years ago

Same here. I don't code in Java so I've been struggling to pinpoint the exact problem, but I've found that these four work alright —

[2020-05-25 15:44:04,256] INFO Added alias 'ChangeTopicCase' to plugin 'com.github.jcustenborder.kafka.connect.transform.common.ChangeTopicCase' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 15:44:04,257] INFO Added alias 'HeaderToField' to plugin 'com.github.jcustenborder.kafka.connect.transform.common.HeaderToField' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 15:44:04,257] INFO Added alias 'SetMaximumPrecision' to plugin 'com.github.jcustenborder.kafka.connect.transform.common.SetMaximumPrecision' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 15:44:04,258] INFO Added alias 'TimestampNow' to plugin 'com.github.jcustenborder.kafka.connect.transform.common.TimestampNow' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)

Apparently one of them implements Transformation, and three others implement BaseKeyValueTransformation, which is nowhere to be found, and yet, those four transformations work 🤷🏼‍♂️.

jcustenborder commented 4 years ago

@volodymyr-kushnir How did you install this? You need to ensure that the dependencies are there as well. If you copied a single jar you're missing all of the dependencies. This is the easiest way to is to use the Confluent Hub

volodymyr-kushnir commented 4 years ago

That's what I did. Here's my Dockerfile

FROM confluentinc/cp-kafka-connect-base:5.5.0
RUN confluent-hub install --no-prompt jcustenborder/kafka-connect-transform-common:latest

I tried different versions instead of "latest" and then I just gave up and tried this instead —

"transforms": "renameFields",
"transforms.renameFields.type": "org.apache.kafka.connect.transforms.ReplaceField$Value",
"transforms.renameFields.renames": "renameThis:rename_this",

— and it still said that "Class could not be found". Then I downgraded to confluentinc/cp-kafka-connect-base:5.0.0 and it still won't work.

jcustenborder commented 4 years ago

Let me give it a look.

volodymyr-kushnir commented 4 years ago

Thanks. Maybe this will help — I see it adding the plugins and aliases, but for some reason it omits adding the aliases for certain plugins, not only yours, but also its own (ReplaceField) as well. I guess I just have to figure out the last version of Kafka Connect where all of these would work 😔.

[2020-05-25 16:48:14,741] INFO Scanning for plugin classes. This might take a moment ... (org.apache.kafka.connect.cli.ConnectDistributed)
[2020-05-25 16:48:14,790] INFO Loading plugin from: /usr/share/confluent-hub-components/jcustenborder-kafka-connect-transform-common (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,426] INFO Registered loader: PluginClassLoader{pluginLocation=file:/usr/share/confluent-hub-components/jcustenborder-kafka-connect-transform-common/} (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,426] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.HeaderToField' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,426] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.BytesToString$Value' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,426] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.ToJSON$Key' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,426] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.PatternFilter$Key' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,426] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.ToLong$Key' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,427] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.BytesToString$Key' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,427] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.ExtractTimestamp$Value' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,427] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.TopicNameToField$Key' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,427] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.ChangeCase$Value' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,427] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.PatternFilter$Value' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,427] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.SetMaximumPrecision$Key' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,427] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.PatternRename$Value' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,427] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.SetMaximumPrecision' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,427] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.ExtractNestedField$Value' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,427] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.TopicNameToField$Value' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,428] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.HeaderToField$Key' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,428] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.ChangeTopicCase' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,428] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.TimestampNow' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,428] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.SchemaNameToTopic$Key' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,428] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.NormalizeSchema$Key' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,428] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.ToLong$Value' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,428] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.PatternRename$Key' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,429] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.SetNull$Value' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,429] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.ExtractNestedField$Key' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,429] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.ChangeCase$Key' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,429] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.SetMaximumPrecision$Value' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,429] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.NormalizeSchema$Value' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,430] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.ExtractTimestamp$Key' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,430] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.HeaderToField$Value' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,430] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.SetNull$Key' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,430] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.ToJSON$Value' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,430] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.TimestampNowField$Value' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,430] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.SchemaNameToTopic$Value' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,431] INFO Added plugin 'com.github.jcustenborder.kafka.connect.transform.common.TimestampNowField$Key' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,431] INFO Added plugin 'org.apache.kafka.connect.connector.policy.AllConnectorClientConfigOverridePolicy' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,431] INFO Added plugin 'org.apache.kafka.connect.connector.policy.PrincipalConnectorClientConfigOverridePolicy' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:15,432] INFO Added plugin 'org.apache.kafka.connect.connector.policy.NoneConnectorClientConfigOverridePolicy' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,152] INFO Registered loader: sun.misc.Launcher$AppClassLoader@764c12b6 (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,152] INFO Added plugin 'org.apache.kafka.connect.mirror.MirrorSourceConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,152] INFO Added plugin 'io.confluent.connect.jdbc.JdbcSinkConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,153] INFO Added plugin 'org.apache.kafka.connect.file.FileStreamSinkConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,153] INFO Added plugin 'io.confluent.connect.jdbc.JdbcSourceConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,153] INFO Added plugin 'org.apache.kafka.connect.file.FileStreamSourceConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,153] INFO Added plugin 'org.apache.kafka.connect.tools.MockSourceConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,153] INFO Added plugin 'org.apache.kafka.connect.tools.SchemaSourceConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,153] INFO Added plugin 'org.apache.kafka.connect.mirror.MirrorCheckpointConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,153] INFO Added plugin 'org.apache.kafka.connect.tools.VerifiableSinkConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,153] INFO Added plugin 'org.apache.kafka.connect.tools.VerifiableSourceConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,153] INFO Added plugin 'org.apache.kafka.connect.tools.MockSinkConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,153] INFO Added plugin 'org.apache.kafka.connect.mirror.MirrorHeartbeatConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,153] INFO Added plugin 'org.apache.kafka.connect.tools.MockConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,153] INFO Added plugin 'org.apache.kafka.connect.converters.FloatConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,153] INFO Added plugin 'org.apache.kafka.connect.converters.DoubleConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,153] INFO Added plugin 'io.confluent.connect.avro.AvroConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,154] INFO Added plugin 'org.apache.kafka.connect.converters.ByteArrayConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,154] INFO Added plugin 'org.apache.kafka.connect.converters.LongConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,154] INFO Added plugin 'org.apache.kafka.connect.converters.IntegerConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,154] INFO Added plugin 'io.confluent.connect.json.JsonSchemaConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,154] INFO Added plugin 'io.confluent.connect.protobuf.ProtobufConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,154] INFO Added plugin 'org.apache.kafka.connect.json.JsonConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,154] INFO Added plugin 'org.apache.kafka.connect.storage.StringConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,154] INFO Added plugin 'org.apache.kafka.connect.converters.ShortConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,154] INFO Added plugin 'org.apache.kafka.connect.storage.SimpleHeaderConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,154] INFO Added plugin 'org.apache.kafka.connect.transforms.ReplaceField$Value' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,154] INFO Added plugin 'org.apache.kafka.connect.transforms.SetSchemaMetadata$Value' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,155] INFO Added plugin 'org.apache.kafka.connect.transforms.ReplaceField$Key' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,155] INFO Added plugin 'org.apache.kafka.connect.transforms.InsertField$Value' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,155] INFO Added plugin 'org.apache.kafka.connect.transforms.TimestampConverter$Key' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,155] INFO Added plugin 'org.apache.kafka.connect.transforms.MaskField$Value' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,155] INFO Added plugin 'org.apache.kafka.connect.transforms.TimestampRouter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,155] INFO Added plugin 'org.apache.kafka.connect.transforms.RegexRouter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,155] INFO Added plugin 'org.apache.kafka.connect.transforms.HoistField$Value' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,155] INFO Added plugin 'org.apache.kafka.connect.transforms.ValueToKey' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,155] INFO Added plugin 'org.apache.kafka.connect.transforms.MaskField$Key' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,156] INFO Added plugin 'org.apache.kafka.connect.transforms.Cast$Key' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,156] INFO Added plugin 'org.apache.kafka.connect.transforms.Cast$Value' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,156] INFO Added plugin 'org.apache.kafka.connect.transforms.ExtractField$Key' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,156] INFO Added plugin 'org.apache.kafka.connect.transforms.Flatten$Value' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,156] INFO Added plugin 'org.apache.kafka.connect.transforms.InsertField$Key' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,156] INFO Added plugin 'org.apache.kafka.connect.transforms.Flatten$Key' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,156] INFO Added plugin 'org.apache.kafka.connect.transforms.SetSchemaMetadata$Key' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,156] INFO Added plugin 'org.apache.kafka.connect.transforms.ExtractField$Value' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,157] INFO Added plugin 'org.apache.kafka.connect.transforms.TimestampConverter$Value' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,157] INFO Added plugin 'org.apache.kafka.connect.transforms.HoistField$Key' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,157] INFO Added plugin 'org.apache.kafka.common.config.provider.FileConfigProvider' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,157] INFO Added plugin 'org.apache.kafka.connect.rest.basic.auth.extension.BasicAuthSecurityRestExtension' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,159] INFO Added aliases 'JdbcSinkConnector' and 'JdbcSink' to plugin 'io.confluent.connect.jdbc.JdbcSinkConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,160] INFO Added aliases 'JdbcSourceConnector' and 'JdbcSource' to plugin 'io.confluent.connect.jdbc.JdbcSourceConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,160] INFO Added aliases 'FileStreamSinkConnector' and 'FileStreamSink' to plugin 'org.apache.kafka.connect.file.FileStreamSinkConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,160] INFO Added aliases 'FileStreamSourceConnector' and 'FileStreamSource' to plugin 'org.apache.kafka.connect.file.FileStreamSourceConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,160] INFO Added aliases 'MirrorCheckpointConnector' and 'MirrorCheckpoint' to plugin 'org.apache.kafka.connect.mirror.MirrorCheckpointConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,160] INFO Added aliases 'MirrorHeartbeatConnector' and 'MirrorHeartbeat' to plugin 'org.apache.kafka.connect.mirror.MirrorHeartbeatConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,160] INFO Added aliases 'MirrorSourceConnector' and 'MirrorSource' to plugin 'org.apache.kafka.connect.mirror.MirrorSourceConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,161] INFO Added aliases 'MockConnector' and 'Mock' to plugin 'org.apache.kafka.connect.tools.MockConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,161] INFO Added aliases 'MockSinkConnector' and 'MockSink' to plugin 'org.apache.kafka.connect.tools.MockSinkConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,161] INFO Added aliases 'MockSourceConnector' and 'MockSource' to plugin 'org.apache.kafka.connect.tools.MockSourceConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,161] INFO Added aliases 'SchemaSourceConnector' and 'SchemaSource' to plugin 'org.apache.kafka.connect.tools.SchemaSourceConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,161] INFO Added aliases 'VerifiableSinkConnector' and 'VerifiableSink' to plugin 'org.apache.kafka.connect.tools.VerifiableSinkConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,161] INFO Added aliases 'VerifiableSourceConnector' and 'VerifiableSource' to plugin 'org.apache.kafka.connect.tools.VerifiableSourceConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,162] INFO Added aliases 'AvroConverter' and 'Avro' to plugin 'io.confluent.connect.avro.AvroConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,162] INFO Added aliases 'JsonSchemaConverter' and 'JsonSchema' to plugin 'io.confluent.connect.json.JsonSchemaConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,162] INFO Added aliases 'ProtobufConverter' and 'Protobuf' to plugin 'io.confluent.connect.protobuf.ProtobufConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,162] INFO Added aliases 'ByteArrayConverter' and 'ByteArray' to plugin 'org.apache.kafka.connect.converters.ByteArrayConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,162] INFO Added aliases 'DoubleConverter' and 'Double' to plugin 'org.apache.kafka.connect.converters.DoubleConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,162] INFO Added aliases 'FloatConverter' and 'Float' to plugin 'org.apache.kafka.connect.converters.FloatConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,163] INFO Added aliases 'IntegerConverter' and 'Integer' to plugin 'org.apache.kafka.connect.converters.IntegerConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,163] INFO Added aliases 'LongConverter' and 'Long' to plugin 'org.apache.kafka.connect.converters.LongConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,163] INFO Added aliases 'ShortConverter' and 'Short' to plugin 'org.apache.kafka.connect.converters.ShortConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,163] INFO Added aliases 'JsonConverter' and 'Json' to plugin 'org.apache.kafka.connect.json.JsonConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,163] INFO Added aliases 'StringConverter' and 'String' to plugin 'org.apache.kafka.connect.storage.StringConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,163] INFO Added aliases 'ByteArrayConverter' and 'ByteArray' to plugin 'org.apache.kafka.connect.converters.ByteArrayConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,163] INFO Added aliases 'DoubleConverter' and 'Double' to plugin 'org.apache.kafka.connect.converters.DoubleConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,164] INFO Added aliases 'FloatConverter' and 'Float' to plugin 'org.apache.kafka.connect.converters.FloatConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,164] INFO Added aliases 'IntegerConverter' and 'Integer' to plugin 'org.apache.kafka.connect.converters.IntegerConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,164] INFO Added aliases 'LongConverter' and 'Long' to plugin 'org.apache.kafka.connect.converters.LongConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,164] INFO Added aliases 'ShortConverter' and 'Short' to plugin 'org.apache.kafka.connect.converters.ShortConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,164] INFO Added aliases 'JsonConverter' and 'Json' to plugin 'org.apache.kafka.connect.json.JsonConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,164] INFO Added alias 'SimpleHeaderConverter' to plugin 'org.apache.kafka.connect.storage.SimpleHeaderConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,164] INFO Added aliases 'StringConverter' and 'String' to plugin 'org.apache.kafka.connect.storage.StringConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,165] INFO Added alias 'ChangeTopicCase' to plugin 'com.github.jcustenborder.kafka.connect.transform.common.ChangeTopicCase' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,166] INFO Added alias 'HeaderToField' to plugin 'com.github.jcustenborder.kafka.connect.transform.common.HeaderToField' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,167] INFO Added alias 'SetMaximumPrecision' to plugin 'com.github.jcustenborder.kafka.connect.transform.common.SetMaximumPrecision' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,167] INFO Added alias 'TimestampNow' to plugin 'com.github.jcustenborder.kafka.connect.transform.common.TimestampNow' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,168] INFO Added alias 'RegexRouter' to plugin 'org.apache.kafka.connect.transforms.RegexRouter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,169] INFO Added alias 'TimestampRouter' to plugin 'org.apache.kafka.connect.transforms.TimestampRouter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,169] INFO Added alias 'ValueToKey' to plugin 'org.apache.kafka.connect.transforms.ValueToKey' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,169] INFO Added alias 'BasicAuthSecurityRestExtension' to plugin 'org.apache.kafka.connect.rest.basic.auth.extension.BasicAuthSecurityRestExtension' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,169] INFO Added aliases 'AllConnectorClientConfigOverridePolicy' and 'All' to plugin 'org.apache.kafka.connect.connector.policy.AllConnectorClientConfigOverridePolicy' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,169] INFO Added aliases 'NoneConnectorClientConfigOverridePolicy' and 'None' to plugin 'org.apache.kafka.connect.connector.policy.NoneConnectorClientConfigOverridePolicy' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
[2020-05-25 16:48:19,170] INFO Added aliases 'PrincipalConnectorClientConfigOverridePolicy' and 'Principal' to plugin 'org.apache.kafka.connect.connector.policy.PrincipalConnectorClientConfigOverridePolicy' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
jcustenborder commented 4 years ago

@volodymyr-kushnir Did you cut that log together or is that one continuous log entry? I'm seeing my plugin loaded with the Apache Kafka Plugins as well. What do you have as the setting for plugin.path? I ran your docker-compose and your config. It worked and loaded the transform. What ENV settings are you launching the container with?

volodymyr-kushnir commented 4 years ago

That was continuous output, preceeded by this —

===> Launching ...
===> Launching kafka-connect ...
[2020-05-25 16:48:14,718] INFO WorkerInfo values:
jvm.args = -Xms256M, -Xmx2G, -XX:+UseG1GC, -XX:MaxGCPauseMillis=20, -XX:InitiatingHeapOccupancyPercent=35, -XX:+ExplicitGCInvokesConcurrent, -XX:MaxInlineLevel=15, -Djava.awt.headless=true, -Dcom.sun.management.jmxremote=true, -Dcom.sun.management.jmxremote.authenticate=false, -Dcom.sun.management.jmxremote.ssl=false, -Dkafka.logs.dir=/var/log/kafka, -Dlog4j.configuration=file:/etc/kafka/connect-log4j.properties
jvm.spec = Azul Systems, Inc., OpenJDK 64-Bit Server VM, 1.8.0_212, 25.212-b04
jvm.classpath = /etc/kafka-connect/jars/*:/usr/share/java/kafka/jersey-container-servlet-core-2.28.jar:/usr/share/java/kafka/jetty-util-9.4.24.v20191120.jar:/usr/share/java/kafka/jakarta.annotation-api-1.3.4.jar:/usr/share/java/kafka/scala-library-2.12.10.jar:/usr/share/java/kafka/commons-compress-1.19.jar:/usr/share/java/kafka/commons-lang3-3.8.1.jar:/usr/share/java/kafka/scala-java8-compat_2.12-0.9.0.jar:/usr/share/java/kafka/aopalliance-repackaged-2.5.0.jar:/usr/share/java/kafka/connect-basic-auth-extension-5.5.0-ccs.jar:/usr/share/java/kafka/kafka-streams-examples-5.5.0-ccs.jar:/usr/share/java/kafka/kafka.jar:/usr/share/java/kafka/jakarta.ws.rs-api-2.1.5.jar:/usr/share/java/kafka/jetty-io-9.4.24.v20191120.jar:/usr/share/java/kafka/metrics-core-2.2.0.jar:/usr/share/java/kafka/avro-1.9.2.jar:/usr/share/java/kafka/netty-handler-4.1.45.Final.jar:/usr/share/java/kafka/commons-logging-1.2.jar:/usr/share/java/kafka/jetty-servlets-9.4.24.v20191120.jar:/usr/share/java/kafka/httpmime-4.5.11.jar:/usr/share/java/kafka/connect-mirror-client-5.5.0-ccs.jar:/usr/share/java/kafka/kafka_2.12-5.5.0-ccs-javadoc.jar:/usr/share/java/kafka/jackson-databind-2.10.2.jar:/usr/share/java/kafka/scala-logging_2.12-3.9.2.jar:/usr/share/java/kafka/plexus-utils-3.2.1.jar:/usr/share/java/kafka/hk2-locator-2.5.0.jar:/usr/share/java/kafka/connect-json-5.5.0-ccs.jar:/usr/share/java/kafka/jersey-client-2.28.jar:/usr/share/java/kafka/reflections-0.9.12.jar:/usr/share/java/kafka/zstd-jni-1.4.4-7.jar:/usr/share/java/kafka/connect-mirror-5.5.0-ccs.jar:/usr/share/java/kafka/kafka-log4j-appender-5.5.0-ccs.jar:/usr/share/java/kafka/connect-file-5.5.0-ccs.jar:/usr/share/java/kafka/netty-common-4.1.45.Final.jar:/usr/share/java/kafka/audience-annotations-0.5.0.jar:/usr/share/java/kafka/log4j-1.2.17.jar:/usr/share/java/kafka/lz4-java-1.7.1.jar:/usr/share/java/kafka/kafka-streams-scala_2.12-5.5.0-ccs.jar:/usr/share/java/kafka/jackson-module-paranamer-2.10.2.jar:/usr/share/java/kafka/support-metrics-client-5.5.0-ccs.jar:/usr/share/java/kafka/jackson-dataformat-csv-2.10.2.jar:/usr/share/java/kafka/jackson-module-jaxb-annotations-2.10.2.jar:/usr/share/java/kafka/argparse4j-0.7.0.jar:/usr/share/java/kafka/netty-codec-4.1.45.Final.jar:/usr/share/java/kafka/zookeeper-jute-3.5.7.jar:/usr/share/java/kafka/netty-transport-native-unix-common-4.1.45.Final.jar:/usr/share/java/kafka/kafka_2.12-5.5.0-ccs-scaladoc.jar:/usr/share/java/kafka/jackson-module-scala_2.12-2.10.2.jar:/usr/share/java/kafka/kafka_2.12-5.5.0-ccs-test-sources.jar:/usr/share/java/kafka/jersey-container-servlet-2.28.jar:/usr/share/java/kafka/commons-codec-1.11.jar:/usr/share/java/kafka/kafka_2.12-5.5.0-ccs-sources.jar:/usr/share/java/kafka/slf4j-api-1.7.30.jar:/usr/share/java/kafka/netty-resolver-4.1.45.Final.jar:/usr/share/java/kafka/jakarta.activation-api-1.2.1.jar:/usr/share/java/kafka/maven-artifact-3.6.3.jar:/usr/share/java/kafka/jackson-datatype-jdk8-2.10.2.jar:/usr/share/java/kafka/httpcore-4.4.13.jar:/usr/share/java/kafka/javax.ws.rs-api-2.1.1.jar:/usr/share/java/kafka/hk2-api-2.5.0.jar:/usr/share/java/kafka/jakarta.inject-2.5.0.jar:/usr/share/java/kafka/osgi-resource-locator-1.0.1.jar:/usr/share/java/kafka/activation-1.1.1.jar:/usr/share/java/kafka/zookeeper-3.5.7.jar:/usr/share/java/kafka/scala-reflect-2.12.10.jar:/usr/share/java/kafka/rocksdbjni-5.18.3.jar:/usr/share/java/kafka/jaxb-api-2.3.0.jar:/usr/share/java/kafka/commons-cli-1.4.jar:/usr/share/java/kafka/connect-transforms-5.5.0-ccs.jar:/usr/share/java/kafka/kafka_2.12-5.5.0-ccs.jar:/usr/share/java/kafka/jersey-common-2.28.jar:/usr/share/java/kafka/jersey-hk2-2.28.jar:/usr/share/java/kafka/jetty-continuation-9.4.24.v20191120.jar:/usr/share/java/kafka/jackson-core-2.10.2.jar:/usr/share/java/kafka/javassist-3.22.0-CR2.jar:/usr/share/java/kafka/javax.servlet-api-3.1.0.jar:/usr/share/java/kafka/jetty-http-9.4.24.v20191120.jar:/usr/share/java/kafka/validation-api-2.0.1.Final.jar:/usr/share/java/kafka/support-metrics-common-5.5.0-ccs.jar:/usr/share/java/kafka/jersey-media-jaxb-2.28.jar:/usr/share/java/kafka/kafka-clients-5.5.0-ccs.jar:/usr/share/java/kafka/jackson-jaxrs-json-provider-2.10.2.jar:/usr/share/java/kafka/scala-collection-compat_2.12-2.1.3.jar:/usr/share/java/kafka/kafka-tools-5.5.0-ccs.jar:/usr/share/java/kafka/javassist-3.26.0-GA.jar:/usr/share/java/kafka/jopt-simple-5.0.4.jar:/usr/share/java/kafka/connect-api-5.5.0-ccs.jar:/usr/share/java/kafka/paranamer-2.8.jar:/usr/share/java/kafka/snappy-java-1.1.7.3.jar:/usr/share/java/kafka/jetty-security-9.4.24.v20191120.jar:/usr/share/java/kafka/connect-runtime-5.5.0-ccs.jar:/usr/share/java/kafka/jetty-client-9.4.24.v20191120.jar:/usr/share/java/kafka/hk2-utils-2.5.0.jar:/usr/share/java/kafka/jakarta.xml.bind-api-2.3.2.jar:/usr/share/java/kafka/netty-buffer-4.1.45.Final.jar:/usr/share/java/kafka/kafka-streams-5.5.0-ccs.jar:/usr/share/java/kafka/jersey-server-2.28.jar:/usr/share/java/kafka/jetty-server-9.4.24.v20191120.jar:/usr/share/java/kafka/kafka_2.12-5.5.0-ccs-test.jar:/usr/share/java/kafka/jackson-annotations-2.10.2.jar:/usr/share/java/kafka/netty-transport-native-epoll-4.1.45.Final.jar:/usr/share/java/kafka/kafka-streams-test-utils-5.5.0-ccs.jar:/usr/share/java/kafka/httpclient-4.5.11.jar:/usr/share/java/kafka/jetty-servlet-9.4.24.v20191120.jar:/usr/share/java/kafka/slf4j-log4j12-1.7.30.jar:/usr/share/java/kafka/netty-transport-4.1.45.Final.jar:/usr/share/java/kafka/jackson-jaxrs-base-2.10.2.jar:/usr/share/java/kafka/kafka-connect-jdbc-6.0.0-SNAPSHOT.jar:/usr/share/java/kafka/slf4j-api-1.7.26.jar:/usr/share/java/kafka/sqlite-jdbc-3.25.2.jar:/usr/share/java/kafka/postgresql-42.2.10.jar:/usr/share/java/kafka/jtds-1.3.1.jar:/usr/share/java/kafka/common-utils-6.0.0-SNAPSHOT.jar:/usr/share/java/kafka/confluent-metrics-5.5.0-ce.jar:/usr/share/java/confluent-common/common-metrics-5.5.0.jar:/usr/share/java/confluent-common/slf4j-api-1.7.26.jar:/usr/share/java/confluent-common/build-tools-5.5.0.jar:/usr/share/java/confluent-common/common-utils-5.5.0.jar:/usr/share/java/confluent-common/common-config-5.5.0.jar:/usr/share/java/kafka-serde-tools/kafka-avro-serializer-5.5.0.jar:/usr/share/java/kafka-serde-tools/wire-runtime-3.1.0.jar:/usr/share/java/kafka-serde-tools/jackson-datatype-joda-2.10.2.jar:/usr/share/java/kafka-serde-tools/kafka-connect-avro-data-5.5.0.jar:/usr/share/java/kafka-serde-tools/scala-library-2.12.10.jar:/usr/share/java/kafka-serde-tools/commons-compress-1.19.jar:/usr/share/java/kafka-serde-tools/commons-lang3-3.8.1.jar:/usr/share/java/kafka-serde-tools/classgraph-4.8.21.jar:/usr/share/java/kafka-serde-tools/jakarta.inject-2.6.1.jar:/usr/share/java/kafka-serde-tools/avro-1.9.2.jar:/usr/share/java/kafka-serde-tools/jersey-client-2.30.jar:/usr/share/java/kafka-serde-tools/jersey-common-2.30.jar:/usr/share/java/kafka-serde-tools/kafka-streams-avro-serde-5.5.0.jar:/usr/share/java/kafka-serde-tools/kotlin-scripting-compiler-embeddable-1.3.50.jar:/usr/share/java/kafka-serde-tools/commons-logging-1.2.jar:/usr/share/java/kafka-serde-tools/failureaccess-1.0.1.jar:/usr/share/java/kafka-serde-tools/swagger-core-1.5.3.jar:/usr/share/java/kafka-serde-tools/jboss-logging-3.3.2.Final.jar:/usr/share/java/kafka-serde-tools/jackson-databind-2.10.2.jar:/usr/share/java/kafka-serde-tools/mbknor-jackson-jsonschema_2.12-1.0.36.jar:/usr/share/java/kafka-serde-tools/protobuf-java-util-3.11.4.jar:/usr/share/java/kafka-serde-tools/kotlin-stdlib-jdk8-1.3.61.jar:/usr/share/java/kafka-serde-tools/j2objc-annotations-1.3.jar:/usr/share/java/kafka-serde-tools/kafka-connect-protobuf-converter-5.5.0.jar:/usr/share/java/kafka-serde-tools/protobuf-java-3.11.4.jar:/usr/share/java/kafka-serde-tools/kafka-protobuf-provider-5.5.0.jar:/usr/share/java/kafka-serde-tools/checker-compat-qual-2.5.5.jar:/usr/share/java/kafka-serde-tools/commons-digester-1.8.1.jar:/usr/share/java/kafka-serde-tools/commons-collections-3.2.2.jar:/usr/share/java/kafka-serde-tools/json-20190722.jar:/usr/share/java/kafka-serde-tools/snakeyaml-1.24.jar:/usr/share/java/kafka-serde-tools/kafka-streams-protobuf-serde-5.5.0.jar:/usr/share/java/kafka-serde-tools/osgi-resource-locator-1.0.3.jar:/usr/share/java/kafka-serde-tools/commons-validator-1.6.jar:/usr/sh
are/java/kafka-serde-tools/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:/usr/share/java/kafka-serde-tools/error_prone_annotations-2.3.4.jar:/usr/share/java/kafka-serde-tools/kafka-connect-json-schema-converter-5.5.0.jar:/usr/share/java/kafka-serde-tools/kotlin-stdlib-jdk7-1.3.61.jar:/usr/share/java/kafka-serde-tools/jersey-media-jaxb-2.30.jar:/usr/share/java/kafka-serde-tools/kotlinx-coroutines-core-1.1.1.jar:/usr/share/java/kafka-serde-tools/jakarta.el-3.0.2.jar:/usr/share/java/kafka-serde-tools/jackson-datatype-jsr310-2.10.2.jar:/usr/share/java/kafka-serde-tools/kotlin-stdlib-common-1.3.61.jar:/usr/share/java/kafka-serde-tools/jakarta.el-api-3.0.3.jar:/usr/share/java/kafka-serde-tools/kafka-json-schema-serializer-5.5.0.jar:/usr/share/java/kafka-serde-tools/kotlin-reflect-1.3.50.jar:/usr/share/java/kafka-serde-tools/jsr305-3.0.2.jar:/usr/share/java/kafka-serde-tools/joda-time-2.10.2.jar:/usr/share/java/kafka-serde-tools/jackson-datatype-jdk8-2.10.2.jar:/usr/share/java/kafka-serde-tools/jakarta.annotation-api-1.3.5.jar:/usr/share/java/kafka-serde-tools/kafka-schema-registry-client-5.5.0.jar:/usr/share/java/kafka-serde-tools/kafka-schema-serializer-5.5.0.jar:/usr/share/java/kafka-serde-tools/handy-uri-templates-2.1.8.jar:/usr/share/java/kafka-serde-tools/guava-28.1-android.jar:/usr/share/java/kafka-serde-tools/kafka-streams-json-schema-serde-5.5.0.jar:/usr/share/java/kafka-serde-tools/hibernate-validator-6.0.17.Final.jar:/usr/share/java/kafka-serde-tools/rocksdbjni-5.18.3.jar:/usr/share/java/kafka-serde-tools/re2j-1.3.jar:/usr/share/java/kafka-serde-tools/gson-2.8.5.jar:/usr/share/java/kafka-serde-tools/jersey-bean-validation-2.30.jar:/usr/share/java/kafka-serde-tools/kafka-protobuf-serializer-5.5.0.jar:/usr/share/java/kafka-serde-tools/jackson-core-2.10.2.jar:/usr/share/java/kafka-serde-tools/swagger-models-1.5.3.jar:/usr/share/java/kafka-serde-tools/kotlinx-coroutines-core-common-1.1.1.jar:/usr/share/java/kafka-serde-tools/kotlin-scripting-common-1.3.50.jar:/usr/share/java/kafka-serde-tools/kafka-connect-avro-converter-5.5.0.jar:/usr/share/java/kafka-serde-tools/classmate-1.3.4.jar:/usr/share/java/kafka-serde-tools/validation-api-2.0.1.Final.jar:/usr/share/java/kafka-serde-tools/annotations-13.0.jar:/usr/share/java/kafka-serde-tools/jersey-server-2.30.jar:/usr/share/java/kafka-serde-tools/jackson-module-parameter-names-2.10.2.jar:/usr/share/java/kafka-serde-tools/animal-sniffer-annotations-1.18.jar:/usr/share/java/kafka-serde-tools/kafka-json-schema-provider-5.5.0.jar:/usr/share/java/kafka-serde-tools/jakarta.validation-api-2.0.2.jar:/usr/share/java/kafka-serde-tools/kotlin-stdlib-1.3.61.jar:/usr/share/java/kafka-serde-tools/jackson-dataformat-yaml-2.10.2.jar:/usr/share/java/kafka-serde-tools/wire-schema-3.1.0.jar:/usr/share/java/kafka-serde-tools/okio-2.4.3.jar:/usr/share/java/kafka-serde-tools/swagger-annotations-1.5.22.jar:/usr/share/java/kafka-serde-tools/kotlin-scripting-compiler-impl-embeddable-1.3.50.jar:/usr/share/java/kafka-serde-tools/kotlin-scripting-jvm-1.3.50.jar:/usr/share/java/kafka-serde-tools/jackson-datatype-guava-2.10.2.jar:/usr/share/java/kafka-serde-tools/jakarta.ws.rs-api-2.1.6.jar:/usr/share/java/kafka-serde-tools/kafka-streams-5.5.0-ccs.jar:/usr/share/java/kafka-serde-tools/kafka-json-serializer-5.5.0.jar:/usr/share/java/kafka-serde-tools/jackson-annotations-2.10.2.jar:/usr/share/java/kafka-serde-tools/org.everit.json.schema-1.12.1.jar:/usr/share/java/kafka-serde-tools/kotlin-script-runtime-1.3.50.jar:/usr/share/java/monitoring-interceptors/monitoring-interceptors-5.5.0.jar:/usr/bin/../share/java/kafka/jersey-container-servlet-core-2.28.jar:/usr/bin/../share/java/kafka/jetty-util-9.4.24.v20191120.jar:/usr/bin/../share/java/kafka/jakarta.annotation-api-1.3.4.jar:/usr/bin/../share/java/kafka/scala-library-2.12.10.jar:/usr/bin/../share/java/kafka/commons-compress-1.19.jar:/usr/bin/../share/java/kafka/commons-lang3-3.8.1.jar:/usr/bin/../share/java/kafka/scala-java8-compat_2.12-0.9.0.jar:/usr/bin/../share/java/kafka/aopalliance-repackaged-2.5.0.jar:/usr/bin/../share/java/kafka/connect-basic-auth-extension-5.5.0-ccs.jar:/usr/bin/../share/java/kafka/kafka-streams-examples-5.5.0-ccs.jar:/usr/bin/../share/java/kafka/kafka.jar:/usr/bin/../share/java/kafka/jakarta.ws.rs-api-2.1.5.jar:/usr/bin/../share/java/kafka/jetty-io-9.4.24.v20191120.jar:/usr/bin/../share/java/kafka/metrics-core-2.2.0.jar:/usr/bin/../share/java/kafka/avro-1.9.2.jar:/usr/bin/../share/java/kafka/netty-handler-4.1.45.Final.jar:/usr/bin/../share/java/kafka/commons-logging-1.2.jar:/usr/bin/../share/java/kafka/jetty-servlets-9.4.24.v20191120.jar:/usr/bin/../share/java/kafka/httpmime-4.5.11.jar:/usr/bin/../share/java/kafka/connect-mirror-client-5.5.0-ccs.jar:/usr/bin/../share/java/kafka/kafka_2.12-5.5.0-ccs-javadoc.jar:/usr/bin/../share/java/kafka/jackson-databind-2.10.2.jar:/usr/bin/../share/java/kafka/scala-logging_2.12-3.9.2.jar:/usr/bin/../share/java/kafka/plexus-utils-3.2.1.jar:/usr/bin/../share/java/kafka/hk2-locator-2.5.0.jar:/usr/bin/../share/java/kafka/connect-json-5.5.0-ccs.jar:/usr/bin/../share/java/kafka/jersey-client-2.28.jar:/usr/bin/../share/java/kafka/reflections-0.9.12.jar:/usr/bin/../share/java/kafka/zstd-jni-1.4.4-7.jar:/usr/bin/../share/java/kafka/connect-mirror-5.5.0-ccs.jar:/usr/bin/../share/java/kafka/kafka-log4j-appender-5.5.0-ccs.jar:/usr/bin/../share/java/kafka/connect-file-5.5.0-ccs.jar:/usr/bin/../share/java/kafka/netty-common-4.1.45.Final.jar:/usr/bin/../share/java/kafka/audience-annotations-0.5.0.jar:/usr/bin/../share/java/kafka/log4j-1.2.17.jar:/usr/bin/../share/java/kafka/lz4-java-1.7.1.jar:/usr/bin/../share/java/kafka/kafka-streams-scala_2.12-5.5.0-ccs.jar:/usr/bin/../share/java/kafka/jackson-module-paranamer-2.10.2.jar:/usr/bin/../share/java/kafka/support-metrics-client-5.5.0-ccs.jar:/usr/bin/../share/java/kafka/jackson-dataformat-csv-2.10.2.jar:/usr/bin/../share/java/kafka/jackson-module-jaxb-annotations-2.10.2.jar:/usr/bin/../share/java/kafka/argparse4j-0.7.0.jar:/usr/bin/../share/java/kafka/netty-codec-4.1.45.Final.jar:/usr/bin/../share/java/kafka/zookeeper-jute-3.5.7.jar:/usr/bin/../share/java/kafka/netty-transport-native-unix-common-4.1.45.Final.jar:/usr/bin/../share/java/kafka/kafka_2.12-5.5.0-ccs-scaladoc.jar:/usr/bin/../share/java/kafka/jackson-module-scala_2.12-2.10.2.jar:/usr/bin/../share/java/kafka/kafka_2.12-5.5.0-ccs-test-sources.jar:/usr/bin/../share/java/kafka/jersey-container-servlet-2.28.jar:/usr/bin/../share/java/kafka/commons-codec-1.11.jar:/usr/bin/../share/java/kafka/kafka_2.12-5.5.0-ccs-sources.jar:/usr/bin/../share/java/kafka/slf4j-api-1.7.30.jar:/usr/bin/../share/java/kafka/netty-resolver-4.1.45.Final.jar:/usr/bin/../share/java/kafka/jakarta.activation-api-1.2.1.jar:/usr/bin/../share/java/kafka/maven-artifact-3.6.3.jar:/usr/bin/../share/java/kafka/jackson-datatype-jdk8-2.10.2.jar:/usr/bin/../share/java/kafka/httpcore-4.4.13.jar:/usr/bin/../share/java/kafka/javax.ws.rs-api-2.1.1.jar:/usr/bin/../share/java/kafka/hk2-api-2.5.0.jar:/usr/bin/../share/java/kafka/jakarta.inject-2.5.0.jar:/usr/bin/../share/java/kafka/osgi-resource-locator-1.0.1.jar:/usr/bin/../share/java/kafka/activation-1.1.1.jar:/usr/bin/../share/java/kafka/zookeeper-3.5.7.jar:/usr/bin/../share/java/kafka/scala-reflect-2.12.10.jar:/usr/bin/../share/java/kafka/rocksdbjni-5.18.3.jar:/usr/bin/../share/java/kafka/jaxb-api-2.3.0.jar:/usr/bin/../share/java/kafka/commons-cli-1.4.jar:/usr/bin/../share/java/kafka/connect-transforms-5.5.0-ccs.jar:/usr/bin/../share/java/kafka/kafka_2.12-5.5.0-ccs.jar:/usr/bin/../share/java/kafka/jersey-common-2.28.jar:/usr/bin/../share/java/kafka/jersey-hk2-2.28.jar:/usr/bin/../share/java/kafka/jetty-continuation-9.4.24.v20191120.jar:/usr/bin/../share/java/kafka/jackson-core-2.10.2.jar:/usr/bin/../share/java/kafka/javassist-3.22.0-CR2.jar:/usr/bin/../share/java/kafka/javax.servlet-api-3.1.0.jar:/usr/bin/../share/java/kafka/jetty-http-9.4.24.v20191120.jar:/usr/bin/../share/java/kafka/validation-api-2.0.1.Final.jar:/usr/bin/../share/java/kafka/support-metrics-common-5.5.0-ccs.jar:/usr/bin/../share/java/kafka/jersey-media-jaxb-2.28.jar:/usr/bin/../share/java/kafka/kafka-clients-5.5.0-ccs.jar:/usr/bin/../
share/java/kaf
ka/jackson-jaxrs-json-provider-2.10.2.jar:/usr/bin/../share/java/kafka/scala-collection-compat_2.12-2.1.3.jar:/usr/bin/../share/java/kafka/kafka-tools-5.5.0-ccs.jar:/usr/bin/../share/java/kafka/javassist-3.26.0-GA.jar:/usr/bin/../share/java/kafka/jopt-simple-5.0.4.jar:/usr/bin/../share/java/kafka/connect-api-5.5.0-ccs.jar:/usr/bin/../share/java/kafka/paranamer-2.8.jar:/usr/bin/../share/java/kafka/snappy-java-1.1.7.3.jar:/usr/bin/../share/java/kafka/jetty-security-9.4.24.v20191120.jar:/usr/bin/../share/java/kafka/connect-runtime-5.5.0-ccs.jar:/usr/bin/../share/java/kafka/jetty-client-9.4.24.v20191120.jar:/usr/bin/../share/java/kafka/hk2-utils-2.5.0.jar:/usr/bin/../share/java/kafka/jakarta.xml.bind-api-2.3.2.jar:/usr/bin/../share/java/kafka/netty-buffer-4.1.45.Final.jar:/usr/bin/../share/java/kafka/kafka-streams-5.5.0-ccs.jar:/usr/bin/../share/java/kafka/jersey-server-2.28.jar:/usr/bin/../share/java/kafka/jetty-server-9.4.24.v20191120.jar:/usr/bin/../share/java/kafka/kafka_2.12-5.5.0-ccs-test.jar:/usr/bin/../share/java/kafka/jackson-annotations-2.10.2.jar:/usr/bin/../share/java/kafka/netty-transport-native-epoll-4.1.45.Final.jar:/usr/bin/../share/java/kafka/kafka-streams-test-utils-5.5.0-ccs.jar:/usr/bin/../share/java/kafka/httpclient-4.5.11.jar:/usr/bin/../share/java/kafka/jetty-servlet-9.4.24.v20191120.jar:/usr/bin/../share/java/kafka/slf4j-log4j12-1.7.30.jar:/usr/bin/../share/java/kafka/netty-transport-4.1.45.Final.jar:/usr/bin/../share/java/kafka/jackson-jaxrs-base-2.10.2.jar:/usr/bin/../share/java/kafka/kafka-connect-jdbc-6.0.0-SNAPSHOT.jar:/usr/bin/../share/java/kafka/slf4j-api-1.7.26.jar:/usr/bin/../share/java/kafka/sqlite-jdbc-3.25.2.jar:/usr/bin/../share/java/kafka/postgresql-42.2.10.jar:/usr/bin/../share/java/kafka/jtds-1.3.1.jar:/usr/bin/../share/java/kafka/common-utils-6.0.0-SNAPSHOT.jar:/usr/bin/../share/java/kafka/confluent-metrics-5.5.0-ce.jar:/usr/bin/../support-metrics-client/build/dependant-libs-2.12/*:/usr/bin/../support-metrics-client/build/libs/*:/usr/share/java/support-metrics-client/*
os.spec = Linux, amd64, 4.19.76-linuxkit
os.vcpus = 6
(org.apache.kafka.connect.runtime.WorkerInfo)

And I run Docker container with these —

docker run -d \
  --name=<name> \
  --publish=80:80 \
  -e CONNECT_BOOTSTRAP_SERVERS="<hostname>:<port>" \
  -e CONNECT_REST_PORT="80" \
  -e CONNECT_GROUP_ID="hive-service-connect" \
  -e CONNECT_CONFIG_STORAGE_TOPIC="<name>" \
  -e CONNECT_OFFSET_STORAGE_TOPIC="<name>" \
  -e CONNECT_STATUS_STORAGE_TOPIC="<name>" \
  -e CONNECT_KEY_CONVERTER="org.apache.kafka.connect.json.JsonConverter" \
  -e CONNECT_VALUE_CONVERTER="org.apache.kafka.connect.json.JsonConverter" \
  -e CONNECT_INTERNAL_KEY_CONVERTER="org.apache.kafka.connect.json.JsonConverter" \
  -e CONNECT_INTERNAL_VALUE_CONVERTER="org.apache.kafka.connect.json.JsonConverter" \
  -e CONNECT_REST_ADVERTISED_HOST_NAME="localhost" \
  -e CONNECT_SSL_ENDPOINT_IDENTIFICATION_ALGORITHM="https" \
  -e CONNECT_SASL_MECHANISM="PLAIN" \
  -e CONNECT_REQUEST_TIMEOUT_MS="20000" \
  -e CONNECT_RETRY_BACKOFF_MS="500" \
  -e CONNECT_SASL_JAAS_CONFIG="<jaas_config>" \
  -e CONNECT_SECURITY_PROTOCOL="SASL_SSL" \
  -e CONNECT_CONSUMER_SSL_ENDPOINT_IDENTIFICATION_ALGORITHM="https" \
  -e CONNECT_CONSUMER_SASL_MECHANISM="PLAIN" \
  -e CONNECT_CONSUMER_REQUEST_TIMEOUT_MS="20000" \
  -e CONNECT_CONSUMER_RETRY_BACKOFF_MS="500" \
  -e CONNECT_CONSUMER_SASL_JAAS_CONFIG="<jaas_config>" \
  -e CONNECT_CONSUMER_SECURITY_PROTOCOL="SASL_SSL" \
  -e CONNECT_PRODUCER_SSL_ENDPOINT_IDENTIFICATION_ALGORITHM="https" \
  -e CONNECT_PRODUCER_SASL_MECHANISM="PLAIN" \
  -e CONNECT_PRODUCER_REQUEST_TIMEOUT_MS="20000" \
  -e CONNECT_PRODUCER_RETRY_BACKOFF_MS="500" \
  -e CONNECT_PRODUCER_SASL_JAAS_CONFIG="<jaas_config>" \
  -e CONNECT_PRODUCER_SECURITY_PROTOCOL="SASL_SSL" \
  -e CONNECT_PLUGIN_PATH=/usr/share/confluent-hub-components \
  name:0.1.0

/usr/share/confluent-hub-components is where /jcustenborder-kafka-connect-transform-common folder lands. lib folder has these —

-rw-r--r-- 1 root root 3.4K May 25 14:19 animal-sniffer-annotations-1.17.jar
-rw-r--r-- 1 root root 189K May 25 14:19 checker-qual-2.5.2.jar
-rw-r--r-- 1 root root 148K May 25 14:19 connect-utils-0.4.162.jar
-rw-r--r-- 1 root root  14K May 25 14:19 error_prone_annotations-2.2.0.jar
-rw-r--r-- 1 root root 4.6K May 25 14:19 failureaccess-1.0.1.jar
-rw-r--r-- 1 root root 1.5M May 25 14:19 freemarker-2.3.25-incubating.jar
-rw-r--r-- 1 root root 2.7M May 25 14:19 guava-27.1-jre.jar
-rw-r--r-- 1 root root 8.6K May 25 14:19 j2objc-annotations-1.1.jar
-rw-r--r-- 1 root root  65K May 25 14:19 jackson-annotations-2.9.0.jar
-rw-r--r-- 1 root root 318K May 25 14:19 jackson-core-2.9.8.jar
-rw-r--r-- 1 root root 1.4M May 25 14:19 jackson-databind-2.9.8.jar
-rw-r--r-- 1 root root  20K May 25 14:19 jsr305-3.0.2.jar
-rw-r--r-- 1 root root 120K May 25 14:19 kafka-connect-transform-common-0.1.0.35.jar
-rw-r--r-- 1 root root 2.2K May 25 14:19 listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar
volodymyr-kushnir commented 4 years ago

Seems like this https://github.com/jcustenborder/kafka-connect-transform-xml/issues/18 is related.

jcustenborder commented 4 years ago

Hmmm weird. This is working for me. Can you give it a shot? The only thing I see in your config is CONNECT_REST_ADVERTISED_HOST_NAME should not be localhost. It should be the hostname of the docker container. I highly doubt that is it though.

version: "3"
services:
  kafka:
    hostname: kafka
    image: confluentinc/cp-enterprise-kafka:5.5.0
    depends_on:
      - zookeeper
    environment:
      KAFKA_ZOOKEEPER_CONNECT: "zookeeper:2181"
      KAFKA_ADVERTISED_LISTENERS: "PLAINTEXT://:9092"
      KAFKA_METRIC_REPORTERS: io.confluent.metrics.reporter.ConfluentMetricsReporter
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      CONFLUENT_METRICS_REPORTER_BOOTSTRAP_SERVERS: localhost:9092
      CONFLUENT_METRICS_REPORTER_ZOOKEEPER_CONNECT: zookeeper:2181
      CONFLUENT_METRICS_REPORTER_TOPIC_REPLICAS: 1
    ports:
      - "9092:9092"
  schema-registry:
    image: confluentinc/cp-schema-registry:5.5.0
    environment:
      SCHEMA_REGISTRY_KAFKASTORE_CONNECTION_URL: "zookeeper:2181"
      SCHEMA_REGISTRY_HOST_NAME: schema-registry
    depends_on:
      - zookeeper
      - kafka
    ports:
      - '8081:8081'
  zookeeper:
    image: confluentinc/cp-zookeeper:5.5.0
    environment:
      ZOOKEEPER_CLIENT_PORT: "2181"
      ZOOKEEPER_TICK_TIME: "2000"
  connect:
    hostname: connect
    build: connect
    depends_on:
    - schema-registry
    - kafka
    ports:
    - '8083:8083'
    environment:
      CONNECT_BOOTSTRAP_SERVERS: "kafka:9092"
      CONNECT_REST_ADVERTISED_HOST_NAME: connect
      CONNECT_PRODUCER_COMPRESSION_TYPE: lz4
      CONNECT_GROUP_ID: connect
      CONNECT_KEY_CONVERTER: io.confluent.connect.avro.AvroConverter
      CONNECT_KEY_CONVERTER_SCHEMA_REGISTRY_URL: http://schema-registry:8081
      CONNECT_VALUE_CONVERTER: io.confluent.connect.avro.AvroConverter
      CONNECT_VALUE_CONVERTER_SCHEMA_REGISTRY_URL: http://schema-registry:8081
      CONNECT_CONFIG_STORAGE_TOPIC: connect_config
      CONNECT_CONFIG_STORAGE_REPLICATION_FACTOR: 1
      CONNECT_OFFSET_STORAGE_TOPIC: connect_offset
      CONNECT_OFFSET_STORAGE_REPLICATION_FACTOR: 1
      CONNECT_STATUS_STORAGE_TOPIC: connect_status
      CONNECT_STATUS_STORAGE_REPLICATION_FACTOR: 1
      CONNECT_INTERNAL_KEY_CONVERTER: org.apache.kafka.connect.json.JsonConverter
      CONNECT_INTERNAL_VALUE_CONVERTER: org.apache.kafka.connect.json.JsonConverter
      CONNECT_INTERNAL_KEY_CONVERTER_SCHEMAS_ENABLE: 'false'
      CONNECT_INTERNAL_VALUE_CONVERTER_SCHEMAS_ENABLE: 'false'
      CONNECT_LOG4J_LOGGERS: 'org.reflections=ERROR'
  control-center:
    image: confluentinc/cp-enterprise-control-center:5.5.0
    depends_on:
    - zookeeper
    - kafka
    - connect
    ports:
    - "9022:9021"
    environment:
      CONTROL_CENTER_SCHEMA_REGISTRY_URL: "http://schema-registry:8081/"
      CONTROL_CENTER_BOOTSTRAP_SERVERS: "kafka:9092"
      CONTROL_CENTER_ZOOKEEPER_CONNECT: "zookeeper:2181"
      CONTROL_CENTER_CONNECT_CLUSTER: 'connect:8083'
      CONTROL_CENTER_REPLICATION_FACTOR: 1
      CONTROL_CENTER_KSQL_ENABLE: "false"

connect/Dockerfile

FROM confluentinc/cp-kafka-connect-base:5.5.0
RUN confluent-hub install --no-prompt jcustenborder/kafka-connect-transform-common:latest
jcustenborder commented 4 years ago

Normally I use something like this and set the plugin path in the container. The one difference I'm seeing is that you're only setting /usr/share/confluent-hub-components and not /usr/share/java,/usr/share/confluent-hub-components to the plugin path.

FROM confluentinc/cp-kafka-connect:5.5.0
ENV CONNECT_PLUGIN_PATH="/usr/share/java,/usr/share/confluent-hub-components"
RUN confluent-hub install --no-prompt jcustenborder/kafka-connect-transform-common:latest
volodymyr-kushnir commented 4 years ago

@jcustenborder thanks so much for your time. I've figured it out. So I followed https://github.com/confluentinc/examples/blob/5.5.0-post/ccloud/connectors/submit_datagen_pageviews_config.sh and the thing is in the shell script it has to be

"transforms.changeCase.type": "com.github.jcustenborder.kafka.connect.transform.common.ChangeCase\$Value",

with a backslash, otherwise it treats $Value as a variable or something and replaces it with an empty string. And then obviously there's no such plugin on Connect 🤷🏼‍♂️. It was screaming in my face the whole time, I just couldn't see it. image Mystery solved, case closed. Again, thank you very much!

jcustenborder commented 4 years ago

Don't feel bad. Issues like these are the hardest to figure out.

ecararus commented 2 years ago

Had similar issue so in case you're using x.properties the type should be: transforms.HoistValue.type= org.apache.kafka.connect.transforms.HoistField\$\Value

also to mention if you still get the "Class could not be found" check the class path or load the dependency i classpath