ismailsimsek
commented
1 year ago @saimigo hive catalog should be configured like below. also added hive-metastore dependency to the release could you try with new release?
debezium.sink.iceberg.type=hive
debezium.sink.iceberg.uri=thrift://hadoop101:9083
debezium.sink.iceberg.clients=5
debezium.sink.iceberg.warehouse=hdfs://example.com:8020/warehouse
debezium.sink.iceberg.hive.metastore.table.owner=xyz
debezium.sink.iceberg.hive.other.configs=xyz
###
debezium.sink.iceberg.engine.hive.enabled=true
### if above one doesn't work please try following config
debezium.sink.iceberg.iceberg.engine.hive.enabled=true
soullinuxer
1、How to configure the use of HiveCatalog and What additional dependencies need to be added, please? 2、How to set table property "engine.hive.enabled=true" when creating tables?
error：
{"timestamp":"2023-05-30T03:14:17.889Z","sequence":100,"loggerClassName":"org.jboss.logging.Logger","loggerName":"io.quarkus.runtime.Application","level":"ERROR","message":"Failed to start application (with profile prod)","threadName":"main","threadId":1,"mdc":{},"ndc":"","hostName":"a85fccd0c6e5","processName":"io.debezium.server.Main","processId":1274,"exception":{"refId":1,"exceptionType":"java.lang.NoSuchMethodException","message":"Cannot find constructor for interface org.apache.iceberg.catalog.Catalog\n\tMissing org.apache.iceberg.hive.HiveCatalog [java.lang.NoClassDefFoundError: org/apache/hadoop/hive/metastore/api/UnknownDBException]","frames":[{"class":"org.apache.iceberg.common.DynConstructors$Builder","method":"buildChecked","line":227},{"class":"org.apache.iceberg.CatalogUtil","method":"loadCatalog","line":180},{"class":"org.apache.iceberg.CatalogUtil","method":"buildIcebergCatalog","line":234},{"class":"io.debezium.server.iceberg.IcebergChangeConsumer","method":"connect","line":128},{"class":"io.debezium.server.iceberg.IcebergChangeConsumer_Bean","method":"create"},{"class":"io.debezium.server.iceberg.IcebergChangeConsumer_Bean","method":"create"},{"class":"io.debezium.server.DebeziumServer","method":"start","line":119},{"class":"io.debezium.server.DebeziumServer_Bean","method":"create"},{"class":"io.debezium.server.DebeziumServer_Bean","method":"create"},{"class":"io.quarkus.arc.impl.AbstractSharedContext","method":"createInstanceHandle","line":111},{"class":"io.quarkus.arc.impl.AbstractSharedContext$1","method":"get","line":35},{"class":"io.quarkus.arc.impl.AbstractSharedContext$1","method":"get","line":32},{"class":"io.quarkus.arc.impl.LazyValue","method":"get","line":26},{"class":"io.quarkus.arc.impl.ComputingCache","method":"computeIfAbsent","line":69},{"class":"io.quarkus.arc.impl.AbstractSharedContext","method":"get","line":32},{"class":"io.quarkus.arc.impl.ClientProxies","method":"getApplicationScopedDelegate","line":18},{"class":"io.debezium.server.DebeziumServer_ClientProxy","method":"arc$delegate"},{"class":"io.debezium.server.DebeziumServer_ClientProxy","method":"arc_contextualInstance"},{"class":"io.debezium.server.DebeziumServer_Observer_Synthetic_d70cd75bf32ab6598217b9a64a8473d65e248c05","method":"notify"},{"class":"io.quarkus.arc.impl.EventImpl$Notifier","method":"notifyObservers","line":323},{"class":"io.quarkus.arc.impl.EventImpl$Notifier","method":"notify","line":305},{"class":"io.quarkus.arc.impl.EventImpl","method":"fire","line":73},{"class":"io.quarkus.arc.runtime.ArcRecorder","method":"fireLifecycleEvent","line":128},{"class":"io.quarkus.arc.runtime.ArcRecorder","method":"handleLifecycleEvents","line":97},{"class":"io.quarkus.deployment.steps.LifecycleEventsBuildStep$startupEvent1144526294","method":"deploy_0"},{"class":"io.quarkus.deployment.steps.LifecycleEventsBuildStep$startupEvent1144526294","method":"deploy"},{"class":"io.quarkus.runner.ApplicationImpl","method":"doStart"},{"class":"io.quarkus.runtime.Application","method":"start","line":101},{"class":"io.quarkus.runtime.ApplicationLifecycleManager","method":"run","line":103},{"class":"io.quarkus.runtime.Quarkus","method":"run","line":67},{"class":"io.quarkus.runtime.Quarkus","method":"run","line":41},{"class":"io.quarkus.runtime.Quarkus","method":"run","line":120},{"class":"io.debezium.server.Main","method":"main","line":15}]}}