search

confluentinc / schema-registry

Confluent Schema Registry for Kafka
https://docs.confluent.io/current/schema-registry/docs/index.html
Other
2.2k stars 1.11k forks source link

Class io.confluent.connect.avro.AvroConverter could not be found #420

Open renukaradhya opened 7 years ago

renukaradhya commented 7 years ago

I am trying to run connect-standalone.bat on windows and getting below error

command: confluent-3.0.1\bin\windows>connect-standalone.bat ../../etc/schema-registry/connect-avro-standalone.properties ../../etc/kafka-connect-elasticsearch/quickstart-elasticsearch.properties

Exception in thread "main" org.apache.kafka.common.config.ConfigException: Invalid value io.confluent.connect.avro.AvroConverter for configuration value.converter: Class io.confluent.connect.avro.AvroConverter could not be found. at org.apache.kafka.common.config.ConfigDef.parseType(ConfigDef.java:672) at org.apache.kafka.common.config.ConfigDef.parse(ConfigDef.java:418) at org.apache.kafka.common.config.AbstractConfig.(AbstractConfig.java:55) at org.apache.kafka.common.config.AbstractConfig.(AbstractConfig.java:62) at org.apache.kafka.connect.runtime.WorkerConfig.(WorkerConfig.java:156) at org.apache.kafka.connect.runtime.standalone.StandaloneConfig.(StandaloneConfig.java:43) at org.apache.kafka.connect.cli.ConnectStandalone.main(ConnectStandalone.java:70)

I build new kafka-connect-avro-converter-3.1.0.jar from \schema-registry-master\avro-converter GIT hub code.

I have placed the kafka-connect-avro-converter-3.1.0.jar in confluent-3.0.1\share\java\schema-registry folder before running the connect-standalone.bat

I even tried creating an instance of AvroConverter in separate project but getting below error: Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/kafka/connect/storage/Converter at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(Unknown Source) at java.security.SecureClassLoader.defineClass(Unknown Source) at java.net.URLClassLoader.defineClass(Unknown Source) at java.net.URLClassLoader.access$100(Unknown Source) at java.net.URLClassLoader$1.run(Unknown Source) at java.net.URLClassLoader$1.run(Unknown Source) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(Unknown Source) at java.lang.ClassLoader.loadClass(Unknown Source) at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source) at java.lang.ClassLoader.loadClass(Unknown Source) at com.targeting.util.MyConverter.main(MyConverter.java:9) Caused by: java.lang.ClassNotFoundException: org.apache.kafka.connect.storage.Converter at java.net.URLClassLoader.findClass(Unknown Source) at java.lang.ClassLoader.loadClass(Unknown Source) at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source) at java.lang.ClassLoader.loadClass(Unknown Source) ... 13 more

jcustenborder commented 7 years ago

@renukaradhya Just a FYI. We are not currently testing the Confluent platform on windows. The class not found is in core Kafka. In general the server processes for Kafka are only tested on Linux.

renukaradhya commented 7 years ago

Hi,

I am using Confluent Platform on windows and fixed some of the issues and checked in to GITHUB. Its available in the below link.

https://github.com/renukaradhya/confluentplatform

Basically updated few batch files.

Regards, Aradhya

On Thu, Oct 13, 2016 at 2:45 AM, Jeremy Custenborder < notifications@github.com> wrote:

@renukaradhya https://github.com/renukaradhya Just a FYI. We are not currently testing the Confluent platform on windows. The class not found is in core Kafka. In general the server processes for Kafka are only tested on Linux.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/confluentinc/schema-registry/issues/420#issuecomment-253341638, or mute the thread https://github.com/notifications/unsubscribe-auth/AVVblKjaiHK6Nq3D9IeM6hw2Sq8WSecFks5qzU4AgaJpZM4KF3TY .

OneCricketeer commented 2 years ago

NoClassDefFoundError: org/apache/kafka/connect/storage/Converter

This is a built-in class with Kafka, and would indicate your classpath for the connect-standalone script was incorrect.

If you still have problems with it, report the problem to Apache Kafka JIRA.

Class io.confluent.connect.avro.AvroConverter could not be found

Nowadays, you should install the Converter from Confluent Hub

https://www.confluent.io/hub/confluentinc/kafka-connect-avro-converter

yalattas commented 1 month ago

I am testing on MAC, and I installed via https://www.confluent.io/hub/confluentinc/kafka-connect-avro-converter but still facing same issue.

I am using confluent operator to perform the installation

apiVersion: platform.confluent.io/v1beta1
kind: Connect
metadata:
  name: kconnect
  namespace: confluent
spec:
  internalTopicReplicationFactor: 1
  oneReplicaPerNode: true
  image:
    application: confluentinc/cp-server-connect:7.7.0
    init: confluentinc/confluent-init-container:2.2.0-1
  build:
    type: onDemand
    onDemand:
      plugins:
        confluentHub:
          - name: debezium-connector-mysql
            owner: debezium
            version: 2.2.1
          - name: kafka-connect-jdbc
            owner: confluentinc
            version: 10.7.4
          - name: kafka-connect-avro-converter
            owner: confluentinc
            version: 7.6.2
  enableSchemas: true

Trace:

config-init-container ===> Installing connector plugins
config-init-container + /mnt/config/install_plugins.sh
config-init-container Running in a "--no-prompt" mode 
...
kconnect [INFO] 2024-07-31 07:41:09,160 [main] org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader installDiscoveredPlugins - Added alias 'AvroConverter' to plugin 'io.confluent.connect.avro.AvroConverter'
kconnect [INFO] 2024-07-31 07:41:09,160 [main] org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader installDiscoveredPlugins - Added alias 'SchemaRegistryConfigProvider' to plugin
...
...
config-init-container Downloading component Kafka Connect Avro Converter 7.6.2, provided by Confluent, Inc. from Confluent Hub and installing into /mnt/plugins 
config-init-container Adding installation directory to plugin path in the following files: 
config-init-container   /dev/null 
config-init-container  
config-init-container Completed 
config-init-container + rm /mnt/config/install_plugins.sh /mnt/config/plugins.json
config-init-container + '[' -n connect ']'
config-init-container + echo 'load script for CP type: connect'
config-init-container + cp -r /opt/confluentinc/connect/ /mnt/config/
config-init-container load script for CP type: connect
kconnect [INFO] 2024-07-31 07:41:09,161 [main] org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader installDiscoveredPlugins - Added alias 'AllConnectorClientConfigOverridePolicy' to plugin 'org.apache.kafka.connect.connector.policy.AllConnectorClientConfigOverridePolicy'
kconnect [INFO] 2024-07-31 07:41:09,161 [main] org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader installDiscoveredPlugins - Added alias 'MirrorSource' to plugin 'org.apache.kafka.connect.mirror.MirrorSourceConnector'
kconnect [INFO] 2024-07-31 07:41:09,161 [main] org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader installDiscoveredPlugins - Added alias 'TopicNameMatches' to plugin 'org.apache.kafka.connect.transforms.predicates.TopicNameMatches'
kconnect [INFO] 2024-07-31 07:41:09,161 [main] org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader installDiscoveredPlugins - Added alias 'ProtobufConverter' to plugin 'io.confluent.connect.protobuf.ProtobufConverter'
kconnect [INFO] 2024-07-31 07:41:09,161 [main] org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader installDiscoveredPlugins - Added alias 'MirrorHeartbeatConnector' to plugin 'org.apache.kafka.connect.mirror.MirrorHeartbeatConnector'
kconnect [INFO] 2024-07-31 07:41:09,161 [main] org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader installDiscoveredPlugins - Added alias 'InsertHeader' to plugin 'org.apache.kafka.connect.transforms.InsertHeader'
kconnect [INFO] 2024-07-31 07:41:09,161 [main] org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader installDiscoveredPlugins - Added alias 'Directory' to plugin 'org.apache.kafka.common.config.provider.DirectoryConfigProvider'
kconnect [INFO] 2024-07-31 07:41:09,161 [main] org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader installDiscoveredPlugins - Added alias 'MirrorSourceConnector' to plugin 'org.apache.kafka.connect.mirror.MirrorSourceConnector'
kconnect [INFO] 2024-07-31 07:41:09,161 [main] org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader installDiscoveredPlugins - Added alias 'PrincipalConnectorClientConfigOverridePolicy' to plugin 'org.apache.kafka.connect.connector.policy.PrincipalConnectorClientConfigOverridePolicy'
kconnect [INFO] 2024-07-31 07:41:09,161 [main] org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader installDiscoveredPlugins - Added alias 'MirrorHeartbeat' to plugin 'org.apache.kafka.connect.mirror.MirrorHeartbeatConnector'
kconnect [INFO] 2024-07-31 07:41:09,161 [main] org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader installDiscoveredPlugins - Added alias 'SecretConfigProvider' to plugin 'io.confluent.kafka.secretregistry.client.config.provider.SecretConfigProvider'
kconnect [INFO] 2024-07-31 07:41:09,161 [main] org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader installDiscoveredPlugins - Added alias 'ValueToKey' to plugin 'org.apache.kafka.connect.transforms.ValueToKey'
kconnect [INFO] 2024-07-31 07:41:09,161 [main] org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader installDiscoveredPlugins - Added alias 'BooleanConverter' to plugin 'org.apache.kafka.connect.converters.BooleanConverter'
kconnect [INFO] 2024-07-31 07:41:09,161 [main] org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader installDiscoveredPlugins - Added alias 'HasHeaderKey' to plugin 'org.apache.kafka.connect.transforms.predicates.HasHeaderKey'
kconnect [INFO] 2024-07-31 07:41:09,161 [main] org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader installDiscoveredPlugins - Added alias 'Integer' to plugin 'org.apache.kafka.connect.converters.IntegerConverter'
kconnect [INFO] 2024-07-31 07:41:09,161 [main] org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader installDiscoveredPlugins - Added alias 'ConnectSecurityExtension' to plugin 'io.confluent.connect.security.ConnectSecurityExtension'
kconnect [INFO] 2024-07-31 07:41:09,161 [main] org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader installDiscoveredPlugins - Added alias 'PreviewRecordTransformer' to plugin 'io.confluent.connect.rest.datapreview.extension.util.PreviewRecordTransformer'
kconnect [INFO] 2024-07-31 07:41:09,162 [main] org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader installDiscoveredPlugins - Added alias 'JsonSchemaConverter' to plugin 'io.confluent.connect.json.JsonSchemaConverter'
kconnect [INFO] 2024-07-31 07:41:09,162 [main] org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader installDiscoveredPlugins - Added alias 'MirrorCheckpoint' to plugin 'org.apache.kafka.connect.mirror.MirrorCheckpointConnector'
kconnect [INFO] 2024-07-31 07:41:09,162 [main] org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader installDiscoveredPlugins - Added alias 'Filter' to plugin 'org.apache.kafka.connect.transforms.Filter'
kconnect [INFO] 2024-07-31 07:41:09,162 [main] org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader installDiscoveredPlugins - Added alias 'None' to plugin 'org.apache.kafka.connect.connector.policy.NoneConnectorClientConfigOverridePolicy'
kconnect [INFO] 2024-07-31 07:41:09,162 [main] org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader installDiscoveredPlugins - Added alias 'SchemaRegistry' to plugin 'io.confluent.kafka.schemaregistry.client.config.provider.SchemaRegistryConfigProvider'
kconnect [INFO] 2024-07-31 07:41:09,162 [main] org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader installDiscoveredPlugins - Added alias 'TimestampRouter' to plugin 'org.apache.kafka.connect.transforms.TimestampRouter'
kconnect [INFO] 2024-07-31 07:41:09,162 [main] org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader installDiscoveredPlugins - Added alias 'Principal' to plugin 'org.apache.kafka.connect.connector.policy.PrincipalConnectorClientConfigOverridePolicy'
kconnect [ERROR] 2024-07-31 07:41:09,199 [main] org.apache.kafka.connect.cli.AbstractConnectCli run - Stopping due to error
kconnect org.apache.kafka.common.config.ConfigException: Invalid value io.confluent.connect.avro.avroconverter for configuration key.converter: Class io.confluent.connect.avro.avroconverter could not be found.
kconnect     at org.apache.kafka.common.config.ConfigDef.parseType(ConfigDef.java:769)
kconnect     at org.apache.kafka.common.config.ConfigDef.parseValue(ConfigDef.java:515)
kconnect     at org.apache.kafka.common.config.ConfigDef.parse(ConfigDef.java:508)
kconnect     at org.apache.kafka.common.config.AbstractConfig.<init>(AbstractConfig.java:116)
kconnect     at org.apache.kafka.connect.runtime.WorkerConfig.<init>(WorkerConfig.java:558)
kconnect     at org.apache.kafka.connect.runtime.distributed.DistributedConfig.<init>(DistributedConfig.java:596)
kconnect     at org.apache.kafka.connect.runtime.distributed.DistributedConfig.<init>(DistributedConfig.java:590)
kconnect     at org.apache.kafka.connect.cli.ConnectDistributed.createConfig(ConnectDistributed.java:111)
kconnect     at org.apache.kafka.connect.cli.ConnectDistributed.createConfig(ConnectDistributed.java:57)
kconnect     at org.apache.kafka.connect.cli.AbstractConnectCli.startConnect(AbstractConnectCli.java:133)
kconnect     at org.apache.kafka.connect.cli.AbstractConnectCli.run(AbstractConnectCli.java:101)
kconnect     at org.apache.kafka.connect.cli.ConnectDistributed.main(ConnectDistributed.java:116)
Stream closed EOF for confluent/kconnect-0 (config-init-container)
Stream closed EOF for confluent/kconnect-0 (kconnect)
rayokota commented 1 month ago

@yalattas , the Avro converter name is case-sensitive, try 

"key.converter": "io.confluent.connect.avro.AvroConverter"

Your error message shows it in all lowercase