search

getindata / kafka-connect-iceberg-sink

Apache License 2.0
77 stars 30 forks source link

Feat/hive metastore #10

Closed sumartoyo closed 2 years ago

sumartoyo commented 2 years ago

Description

To use catalog-impl org.apache.iceberg.hive.HiveCatalog we need to include hive-metastore. Without it, it will throw ClassNotFoundException.

Example config:

{
  "name": "iceberg-sink",
  "config": {
    "connector.class": "com.getindata.kafka.connect.iceberg.sink.IcebergSink",
    ...
    "iceberg.catalog-impl": "org.apache.iceberg.hive.HiveCatalog",
    "iceberg.uri": "thrift://localhost:9083",
    ...
  }
}
PR Checklist
gliter commented 2 years ago

Could I ask you to add example configuration to README.md?

gliter commented 2 years ago

@sumartoyo looks like there is a system test failure, could you take a look on it?

sumartoyo commented 2 years ago

It seems the test issue is because of mandatory topic.prefix is not set. I have added that and the test runs fine now.