databricks / iceberg-kafka-connect

Apache License 2.0
220 stars 49 forks source link

MSK Connect Problem (java.lang.NoSuchMethodError) #311

Open sotiriskar opened 3 days ago

sotiriskar commented 3 days ago

Here's my connector configuration:

{
  "connector.class": "io.tabular.iceberg.connect.IcebergSinkConnector",
  "iceberg.tables.evolve-schema-enabled": "true",
  "iceberg.catalog.catalog-impl": "org.apache.iceberg.aws.glue.GlueCatalog",
  "iceberg.tables.schema-force-optional": "true",
  "tasks.max": "5",
  "topics": "topic.service_borg.device_check",
  "iceberg.catalog.io-impl": "org.apache.iceberg.aws.s3.S3FileIO",
  "iceberg.catalog.client.region": "eu-west-1",
  "iceberg.tables.auto-create-enabled": "true",
  "iceberg.tables": "gatekeeper_analytics_bronze.device_check_msk",
  "iceberg. catalog.warehouse": "s3a://gatekeeper-analytics-bronze/warehouse",
}

I get error:

[Worker-01fa9cd82f93ea100] java.lang.NoSuchMethodError: 'org.apache.kafka.clients.admin.ListConsumerGroupOffsetsOptions org.apache.kafka.clients.admin.ListConsumerGroupOffsetsOptions.requireStable(boolean)'

any idea what might cause this? Is It compatibility issue? I use: Apache Kafka version: 3.6.0 Apache Kafka Connect version: 2.7.1

thanks, in advance.