omry / omegaconf

Flexible Python configuration system. The last one you will ever need.
BSD 3-Clause "New" or "Revised" License
1.88k stars 98 forks source link

Make `select` and `oc.select` more robust #1127

Open SZiesche opened 9 months ago

SZiesche commented 9 months ago

Is your feature request related to a problem? Please describe. By using oc.select we can define default values if the interpolation target is not found. However, the "not found" part is not very robust. For example, if we refer to a non-existing node, we still raise an error.

Consider for example the case, where we have a config, that is usually nested in some other config and has a member like my_member: int = ${oc.select:..member, 5} that is by default inherited from the parent. Now we want to test this config standalone or we want to use it in a different context were the parent doesn't exist, then the code just crashes.

Describe the solution you'd like It would be cool to turn this off or to give the user an option to turn this off. For example, we could change the select implementation to

def select_value(
    cfg: Container,
    key: str,
    *,
    default: Any = _DEFAULT_MARKER_,
    throw_on_resolution_failure: bool = True,
    throw_on_missing: bool = False,
    absolute_key: bool = False,
) -> Any:
    try:  # this try/except block is new, the rest stays the same.
        node = select_node(
            cfg=cfg,
            key=key,
            throw_on_resolution_failure=throw_on_resolution_failure,
            throw_on_missing=throw_on_missing,
            absolute_key=absolute_key,
        )
    except ConfigKeyError as e:
        node = None

    node_not_found = node is None
    if node_not_found or node._is_missing():
        if default is not _DEFAULT_MARKER_:
            return default
        else:
            return None

    return _get_value(node)

Describe alternatives you've considered Alternatively, we could add a flag like throw_on_node_not_existing to the function and raise the error depending on that value. Then the users could relatively easy write their own robust select method.

odelalleau commented 9 months ago

Indeed, it shouldn't crash if it can't find the node when a default value is provided.

SZiesche commented 9 months ago

to make this more precise, I would like to have the following script pass.

from dataclasses import dataclass

from omegaconf import OmegaConf, II

cfg1 = OmegaConf.create({"a": "${oc.select:..member, 5}"})
print(cfg1.a)  # this raises an Error because of the missing node

@dataclass
class Config:
    a: int = II("${oc.select:..member, 5}")

cfg2 = OmegaConf.structured(Config)
print(cfg2.a)  # this raises an Error too