Closed xushiyan closed 2 months ago
i could work on that if no one plan for.
As externalized-config-file described , should scan all files under HUDI_CONF_DIR directory, and maybe filter with *.conf.
And i havn't found the format of this file, maybe properties, ini etc. Just got some ideas from Hudi Issues #5291, like the following.
# Default system properties included when running Hudi jobs.
# This is useful for setting default environmental settings.
# Example:
hoodie.datasource.write.table.type COPY_ON_WRITE
hoodie.datasource.write.hive_style_partitioning false
@gohalo great! as per the linked doc, only hudi-defaults.conf
should be loaded under HUDI_CONF_DIR
. The format should be like a conf file as you showed or delimited by =
like
hoodie.datasource.write.recordkey.field=uuid
During
crates/core/src/table/mod.rs:96 load_configs<I, K, V>
, hudi global configs (ahudi-defaults.conf
file loaded in dir specified byHUDI_CONF_DIR
env var) should be loaded as well.