omry / omegaconf

Flexible Python configuration system. The last one you will ever need.
BSD 3-Clause "New" or "Revised" License
1.94k stars 105 forks source link

[Feature Request] Use SimpleNamespace for to_container() and to_object() #1048

Open raphCode opened 1 year ago

raphCode commented 1 year ago

šŸš€ Feature Request

Since Python 3.3, the standard library provides types.SimpleNamespace which is a container similar to dicts, but can be accessed by dot notation. I would like to have OmegaConf.to_container() / OmegaConf.to_object() emit instances of SimpleNamespace instead of dict.

Motivation

Currently, users can convert OmegaConf objects into python lists and containers for named keys: These can be dataclasses / attr classes (in the case of structured configs) or else dicts. This provides speedups of 3 orders of magnitude when accessing an item compared to OmegaConf objects.

In my code, I want to access the config by dot notation, e.g. config.database.port. While this works great for dataclasses / attr classes, it requires a structured config meaning the presence of all keys must be known in advance. All extra keys not present in the structured config are converted to dict which must be accessed by subscription: config["database"]["port"]

My concrete use case: In my app, users can provide custom implementations of some functions. These functions may read custom options from a hydra config. Both the parameter names and nesting levels of these options are completely user-controlled, so there is no way I can use a structured config in advance.

Pitch

OmegaConf.to_container(cfg, structured_config_mode=omegaconf.SCMode.SIMPLE_NS)

If the idea is well received, OmegaConf.to_object() could also convert to SimpleNamespace by default, but this is a breaking change and not a priority for me.

omry commented 1 year ago

This would break all code that depends on the returned type from to_container being dict. Isn't there a standard way to recursively convert a dict to SimpleNamespace?

raphCode commented 1 year ago

This is why I advocated for hiding this new return type behind structured_config_mode=omegaconf.SCMode.SIMPLE_NS and adding the appropiate Enum key.

I am currently doing this for conversation:

def to_namespace_recurse(x) -> SimpleNamespace:
    if isinstance(x, dict):
        return SimpleNamespace(**{k: to_namespace_recurse(v) for k, v in x.items()})
    if isinstance(x, list):
        return list(map(to_namespace_recurse, x))
    return x