databricks / iceberg-kafka-connect

Apache License 2.0
220 stars 49 forks source link

Support loading Hadoop config files from a specified directory #70

Closed bryanck closed 1 year ago

bryanck commented 1 year ago

This PR adds a new config property, iceberg.hadoop-conf-dir, for specifying a directory containing Hadoop configuration files. If specified, any files named core-site.xml, hdfs-site.xml, or hive-site.xml in this directory will be loaded into the Hadoop configuration (in that order). Hadoop properties in the sink config take precedence. Next precedence are properties loaded from the config directory. Lowest precedence are properties in config files loaded from the classpath.