search

databricks / iceberg-kafka-connect

Apache License 2.0
220 stars 49 forks source link

Kafka connect fails to start due to dependency error on org/apache/iceberg/IcebergBuild class #267

Open abhilater opened 4 months ago

abhilater commented 4 months ago

Steps to reproduce the issue

  1. Download the connector dist eg iceberg-kafka-connect-runtime-0.6.18/
  2. Link the lib path to plugin.path on ./config/connect-distributed-sink.properties eg 
plugin.path=/root/iceberg-kafka-connect-runtime-0.6.18/lib
  1. Start Kafka connect sink 
    cd /opt/kafka/kafka_2.11-2.1.1/
./bin/connect-distributed.sh ./config/connect-distributed-sin.properties

Kafka Connect fails to start with NoClassDefFoundError: org/apache/iceberg/IcebergBuild

Error log 

[2024-06-25 11:15:03,616] ERROR Stopping due to error (org.apache.kafka.connect.cli.ConnectDistributed:84)
java.lang.NoClassDefFoundError: org/apache/iceberg/IcebergBuild
        at io.tabular.iceberg.connect.IcebergSinkConfig.version(IcebergSinkConfig.java:115)
        at io.tabular.iceberg.connect.IcebergSinkConnector.version(IcebergSinkConnector.java:38)
        at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.versionFor(DelegatingClassLoader.java:390)
        at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.versionFor(DelegatingClassLoader.java:395)
        at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.getPluginDesc(DelegatingClassLoader.java:365)
        at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.scanPluginPath(DelegatingClassLoader.java:337)
        at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.scanUrlsAndAddPlugins(DelegatingClassLoader.java:268)
        at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.registerPlugin(DelegatingClassLoader.java:260)
        at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.initPluginLoader(DelegatingClassLoader.java:229)
        at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.initLoaders(DelegatingClassLoader.java:206)
        at org.apache.kafka.connect.runtime.isolation.Plugins.<init>(Plugins.java:61)
        at org.apache.kafka.connect.cli.ConnectDistributed.startConnect(ConnectDistributed.java:91)
        at org.apache.kafka.connect.cli.ConnectDistributed.main(ConnectDistributed.java:78)
Caused by: java.lang.ClassNotFoundException: org.apache.iceberg.IcebergBuild
        at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:476)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:594)
        at org.apache.kafka.connect.runtime.isolation.PluginClassLoader.loadClass(PluginClassLoader.java:104)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527)
        ... 13 more

Need help on resolving this issue

tianyuningmou commented 1 week ago

same issue It seems to be an issue with the order in which plug-ins are loaded, but it hasn’t been resolved yet.