Closed klamann closed 1 year ago
Hello @klamann, thank you for sharing your thoughts, and I'm glad you're finding OmegaConf useful!
A few notes:
OmegaConf.merge
always take precedence over the left argumentsOmegaConf.create
's return type: this has been proposed in https://github.com/omry/omegaconf/issues/899.Your custom wrappers around OmegaConf.create
look reasonable.
Note that the OmegaConf API facilitates working directly with config objects (DictConfig
and ListConfig
) and manipulating the data stored inside of them, but it does not address use cases such as higher-level orchestration of config composition.
Have you seen hydra? Hydra uses OmegaConf as a backend and is designed to handle some of the complexity in orchestrating config composition.
Merging configs and converting untyped configs to structured configs currently takes several function calls. Also, it is not trivial to decide, which functions to use (e.g.
merge
vsunsafe_merge
) and you can shoot yourself in the foot if you apply structured config conversion and merging in the wrong order.Let's say I have two configs: a base config and another config that extends the base config for a different staging environment:
Let's turn this into a
DictConfig
:Three function calls, pretty straightforward.
Now let's convert this to a structured config
here we first have to do the
DictConfig
conversion from before, then specifyMyConfig
in the merge function and finally callto_object
:now we have 4 function calls in a row, and
class_conf
doesn't even have type information, so we have to specify the type hint forMyConfig
ourselves. Also, if we would have resolved eachDictConfig
first and then merged it, the value ofcsv_file
would have been/tmp/users.csv
instead of/opt/users.csv
. And we could have usedunsafe_merge
here for a slight performance boost.This is not trivial to remember, so whenever I use OmegaConf, I carry a bunch of utility functions around. But I think we can slightly extend some OmegaConf functions to make this easier for everyone. This is how I would like to use the
OmegaConf.create
functionthen we could just call
OmegaConf.create(yml_base, yml_prod)
to get the merged config.To directly create a structured config:
now we can call
OmegaConf.create_structured(MyConfig, yml_base, yml_prod)
to get the merged config as instance ofMyConfig
, and we even get the correct type information.We could extend
OmegaConf.load
in the same way and addOmegaConf.load_structured
so we can load structured configs directly from files.What do you think?