omry / omegaconf

Flexible Python configuration system. The last one you will ever need.
BSD 3-Clause "New" or "Revised" License
1.91k stars 104 forks source link

Interpolation doesn't work in subconfigs #1099

Closed MatteoVoges closed 1 year ago

MatteoVoges commented 1 year ago

Describe the bug It seems like, interpolation does not work in sub configs.

To Reproduce

from omegaconf import OmegaConf as oc

src = {
    "parameters": {
        "key": "value",
        "interpolation": "${key}",
    }
}

cfg = oc.create(src)
print(oc.to_yaml(cfg)) # 1

params = cfg.get("parameters")
print(oc.to_yaml(params)) # 2

oc.resolve(params)
print(oc.to_yaml(params)) # 3
parameters:
  key: value
  interpolation: ${key}

key: value
interpolation: ${key}

InterpolationKeyError: Interpolation key 'key' not found

Expected behavior I would except that the interpolation ${key} gets resolved properly into value, because the second config (# 2) is a valid config. Do I have to use another accessor instead of params = cfg.get("parameters") ?

Interactive shell

>>> from omegaconf import OmegaConf as oc
>>> cfg = oc.create({"a":{"b": 1}})
>>> cfg.a.b
1
>>> cfg = cfg.get("a")
>>> cfg.b
1

What is the difference between accessing the elements and resolving the config?

Additional context

MatteoVoges commented 1 year ago

I'm using an obvious work-around at the moment: Instead of params = cfg.get("parameters") I'm using params = oc.create(cfg.parameters) to ensure, that I have a new config object.

Jasha10 commented 1 year ago

@MatteoVoges see the docs on Variable Interpolation, specifically the parts about using relative paths.

src = {
    "parameters": {
        "key": "value",
        "interpolation": "${.key}",
    }
}

Note the change from ${key} to ${.key} above. The interpolation path key would point to a node at the top level of the config, while the path .key is relative. You can also use e.g. ${..key} to point into the parent config.

MatteoVoges commented 1 year ago

Thanks for your answer, but this is not what I meant. With params = cfg.get("parameters") in line 13 in my repro script, it produces the config:

key: value
interpolation: ${key}

I would expect, that this config works like a stand-alone config and ${key} gets resolved to value. I think, that the conflict here is, that any accessing operator (cfg.parameters, cfg.get("parameters")...) returns the data in that key but it is still the same config object.

Maybe we could provide a accessor function that returns a new config object starting from the accessed key. But I guess that my workaround will do it too.

omry commented 1 year ago

Interpolations are absolute by default. Even when you are looking at a node of the config, it has a reference to the config root to facilitate interpolation resolution. If you want to access the config key you should use ${.key} as @Jasha10 pointed out.