search

tabular-io / iceberg-kafka-connect

Apache License 2.0
171 stars 31 forks source link

java.lang.ClassNotFoundException: org.apache.iceberg.IcebergBuild when start connector #213

Closed zhaoli2333 closed 3 months ago

zhaoli2333 commented 3 months ago

I installed this connect by the steps below:

  1. I downloaded iceberg-kafka-connect-runtime-0.6.12.zip and unzip it.
  2. add configure plugin.path=/usr/local/kafka/connectors/iceberg-kafka-connect-runtime-hive-0.6.5/lib to file connect-distributed.properties
  3. startup connect worker using command bin/connect-distributed.sh -daemon config/connect-distributed.properties

and I got this error:

java.lang.NoClassDefFoundError: org/apache/iceberg/IcebergBuild
        at io.tabular.iceberg.connect.IcebergSinkConfig.version(IcebergSinkConfig.java:115)
        at io.tabular.iceberg.connect.IcebergSinkConnector.version(IcebergSinkConnector.java:38)
        at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.versionFor(DelegatingClassLoader.java:387)
        at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.versionFor(DelegatingClassLoader.java:392)
        at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.getPluginDesc(DelegatingClassLoader.java:362)
        at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.scanPluginPath(DelegatingClassLoader.java:334)
        at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.scanUrlsAndAddPlugins(DelegatingClassLoader.java:268)
        at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.registerPlugin(DelegatingClassLoader.java:260)
        at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.initPluginLoader(DelegatingClassLoader.java:229)
        at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.initLoaders(DelegatingClassLoader.java:206)
        at org.apache.kafka.connect.runtime.isolation.Plugins.<init>(Plugins.java:61)
        at org.apache.kafka.connect.cli.ConnectDistributed.startConnect(ConnectDistributed.java:93)
        at org.apache.kafka.connect.cli.ConnectDistributed.main(ConnectDistributed.java:80)
Caused by: java.lang.ClassNotFoundException: org.apache.iceberg.IcebergBuild
        at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:445)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:587)
        at org.apache.kafka.connect.runtime.isolation.PluginClassLoader.loadClass(PluginClassLoader.java:104)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:520)
        ... 13 more
rosner commented 3 months ago

I installed this connect by the steps below:

  1. I downloaded iceberg-kafka-connect-runtime-0.6.12.zip and unzip it.
  2. add configure plugin.path=/usr/local/kafka/connectors/iceberg-kafka-connect-runtime-hive-0.6.5/lib to file connect-distributed.properties

I think there's a mismatch here in what you donwloaded and how your plugin.path is configured:

First your plugin.path contains iceberg-kafka-connect-runtime-hive-0.6.5 which contradicts with the 0.6.12 version. Second if I look at the 0.6.12 release (unzip -l iceberg-kafka-connect-runtime-0.6.12.zip) I get the following:

Archive:  ~/Downloads/iceberg-kafka-connect-runtime-0.6.12.zip
  Length      Date    Time    Name
---------  ---------- -----   ----
        0  02-29-2024 12:49   iceberg-kafka-connect-runtime-0.6.12/
     1313  02-29-2024 12:49   iceberg-kafka-connect-runtime-0.6.12/manifest.json
        0  02-29-2024 12:49   iceberg-kafka-connect-runtime-0.6.12/lib/
    93921  02-29-2024 12:49   iceberg-kafka-connect-runtime-0.6.12/lib/iceberg-kafka-connect-0.6.12.jar
    11189  02-29-2024 12:49   iceberg-kafka-connect-runtime-0.6.12/
...

And you have iceberg-kafka-connect-runtime-hive-0.6.5/lib in your Kafka Connect plugin.path. Again there's a version mismatch but also you have a wrong path there.

Based on the above I think your plugin.path property should look like /usr/local/kafka/connectors/ iceberg-kafka-connect-runtime-0.6.12.