omry / omegaconf

Flexible Python configuration system. The last one you will ever need.
BSD 3-Clause "New" or "Revised" License
1.94k stars 105 forks source link

Allow for automatic `.` separated key conversion to nested dictionary #1188

Open puneeter opened 1 month ago

puneeter commented 1 month ago

Describe the bug

In[25]: oc
Out[25]: {'a.b': 1, 'a.c': 2}

In[26]: OmegaConf.select(oc, "a.b")

In[27]: OmegaConf.select(oc, ".a.b")

In[28]: OmegaConf.select(oc, ".")
Out[28]: {'a.b': 1, 'a.c': 2}

There should be a way to specify . separated namespaces in my config and OmegaConf should ideally split all these keys into nested dictionaries.

To Reproduce See in the above section

Expected behavior

For the above code I expect the following output:

In[26]: OmegaConf.select(oc, "a.b")
Out[26]: {'a.b': 1}

In[27]: OmegaConf.select(oc, ".a.b")
Out[27]: {'a.b': 1}

Additional context

noklam commented 1 month ago

related: #1189

odelalleau commented 1 month ago

There should be a way to specify . separated namespaces in my config and OmegaConf should ideally split all these keys into nested dictionaries.

Currently it is allowed (but not recommended) to use dots in config keys. This makes OmegaConf compatible with a wider range of use cases, and I don't foresee it changing in the future.

That being said, it may be useful to allow creating a config from a dict whose keys contains dot and automatically turning it into a nested config. This couldn't just use the syntax from #1189 as it would break backward compatibility, but this could be either a new parameter to create(), or a new function.

In the meantime, my suggestion is that you implement this function yourself -- you could then share it here so that someone else looking for the same feature can re-use it directly :)

puneeter commented 4 weeks ago

Thanks for responding @odelalleau ! Would you be open to have this functionality embedded in the create method?

odelalleau commented 4 weeks ago

Thanks for responding @odelalleau ! Would you be open to have this functionality embedded in the create method?

Potentially yes, but only if some additional flag is needed to enable this behavior, since we shouldn't break the existing default behavior which is working as intended.

(EDIT: as a result I may lean more towards a different dedicated function, like OmegaConf.create_from_dot_keys() -- since I don't think that a special flag would make sense for the non-dict inputs that OmegaConf.create() currently accepts)

omry commented 2 weeks ago

OmegaConf.select treats "." as a separator. In theory it could be possible to extend it to support escaping of the . (e.g. select(cfg, "a\.b") ), but this is not something that is likely to be included.