This PR updates the method for creating a Hadoop config so it first attempts to instantiate an HdfsConfiguration if it is available (which loads hdfs-site.xml and core-site.xml on the classpath). If the class is not available, then the method will fall back to Configuration which only loads core-site.xml on the classpath. Furthermore, additional Hadoop config properties can be set in the connector config, by prefixing them with iceberg.hadoop.. The readme section on configuration was updated to reflect this.
This PR updates the method for creating a Hadoop config so it first attempts to instantiate an
HdfsConfigurationif it is available (which loadshdfs-site.xmlandcore-site.xmlon the classpath). If the class is not available, then the method will fall back toConfigurationwhich only loadscore-site.xmlon the classpath. Furthermore, additional Hadoop config properties can be set in the connector config, by prefixing them withiceberg.hadoop.. The readme section on configuration was updated to reflect this.