[Bug] Selecting multiple configs from a group should allow selecting none

[bencwallace]

🐛 Bug

Description

When selecting n configs from a group group_name, I would expect the resulting DictConfig corresponding to the group_name key in the composed config to have length n. This seems to be true except in the case n = 0, where the group_name key cannot be found in the composed config at all.

Checklist

• [X] I checked on the latest version of Hydra
• [X] I created a minimal repro (See this for tips).

To reproduce

Minimal Code/Config snippet to reproduce

Here's an element of the fruits config group:

# conf/fruits/banana.yaml
banana:
  color: yellow

Here's the main config:

# conf/config.yaml
defaults:
  - fruits:
    - banana

Here's a Python script:

# main.py
import hydra

@hydra.main(config_path="conf", config_name="config")
def main(cfg):
    print(len(cfg.fruits))

if __name__ == "__main__":
    main()

Running this with python main.py or python main.py fruits=[banana] produces the output 1, as expected. However, running it as HYDRA_FULL_ERROR=1 python main.py fruits=[] produces the following error:

Stack trace/error message

Error executing job with overrides: ['fruits=[]']
Traceback (most recent call last):
  File "06.py", line 8, in <module>
    main()
  File "/home/bwallace/miniconda3/envs/test/lib/python3.7/site-packages/hydra/main.py", line 52, in decorated_main
    config_name=config_name,
  File "/home/bwallace/miniconda3/envs/test/lib/python3.7/site-packages/hydra/_internal/utils.py", line 378, in _run_hydra
    lambda: hydra.run(
  File "/home/bwallace/miniconda3/envs/test/lib/python3.7/site-packages/hydra/_internal/utils.py", line 214, in run_and_report
    raise ex
  File "/home/bwallace/miniconda3/envs/test/lib/python3.7/site-packages/hydra/_internal/utils.py", line 211, in run_and_report
    return func()
  File "/home/bwallace/miniconda3/envs/test/lib/python3.7/site-packages/hydra/_internal/utils.py", line 381, in <lambda>
    overrides=args.overrides,
  File "/home/bwallace/miniconda3/envs/test/lib/python3.7/site-packages/hydra/_internal/hydra.py", line 111, in run
    _ = ret.return_value
  File "/home/bwallace/miniconda3/envs/test/lib/python3.7/site-packages/hydra/core/utils.py", line 233, in return_value
    raise self._return_value
  File "/home/bwallace/miniconda3/envs/test/lib/python3.7/site-packages/hydra/core/utils.py", line 160, in run_job
    ret.return_value = task_function(task_cfg)
  File "06.py", line 5, in main
    print(len(cfg.fruits))
  File "/home/bwallace/miniconda3/envs/test/lib/python3.7/site-packages/omegaconf/dictconfig.py", line 357, in __getattr__
    self._format_and_raise(key=key, value=None, cause=e)
  File "/home/bwallace/miniconda3/envs/test/lib/python3.7/site-packages/omegaconf/base.py", line 196, in _format_and_raise
    type_override=type_override,
  File "/home/bwallace/miniconda3/envs/test/lib/python3.7/site-packages/omegaconf/_utils.py", line 741, in format_and_raise
    _raise(ex, cause)
  File "/home/bwallace/miniconda3/envs/test/lib/python3.7/site-packages/omegaconf/_utils.py", line 719, in _raise
    raise ex.with_traceback(sys.exc_info()[2])  # set end OC_CAUSE=1 for full backtrace
  File "/home/bwallace/miniconda3/envs/test/lib/python3.7/site-packages/omegaconf/dictconfig.py", line 351, in __getattr__
    return self._get_impl(key=key, default_value=_DEFAULT_MARKER_)
  File "/home/bwallace/miniconda3/envs/test/lib/python3.7/site-packages/omegaconf/dictconfig.py", line 438, in _get_impl
    node = self._get_node(key=key, throw_on_missing_key=True)
  File "/home/bwallace/miniconda3/envs/test/lib/python3.7/site-packages/omegaconf/dictconfig.py", line 465, in _get_node
    self._validate_get(key)
  File "/home/bwallace/miniconda3/envs/test/lib/python3.7/site-packages/omegaconf/dictconfig.py", line 167, in _validate_get
    key=key, value=value, cause=ConfigAttributeError(msg)
  File "/home/bwallace/miniconda3/envs/test/lib/python3.7/site-packages/omegaconf/base.py", line 196, in _format_and_raise
    type_override=type_override,
  File "/home/bwallace/miniconda3/envs/test/lib/python3.7/site-packages/omegaconf/_utils.py", line 821, in format_and_raise
    _raise(ex, cause)
  File "/home/bwallace/miniconda3/envs/test/lib/python3.7/site-packages/omegaconf/_utils.py", line 719, in _raise
    raise ex.with_traceback(sys.exc_info()[2])  # set end OC_CAUSE=1 for full backtrace
omegaconf.errors.ConfigAttributeError: Key 'fruits' is not in struct
    full_key: fruits
    object_type=dict

Expected Behavior

I would expect running the last command to produce the output 0.

System information

• Hydra Version : 1.1.1
• Python version : 3.7.11
• Virtual environment type and version : conda 4.9.2
• Operating system : Ubuntu 18.04

Additional context

This isn't that much of a problem when the fruits is only expected by Python code, where input validation can be used. However, it's more of a problem when the key is expected by other parts of the Hydra config, e.g. suppose we had the following somewhere else in the config:

basket:
  fruits: ${fruits}
  cheese: ${cheese}
  ...

Then we need to remove the fruits sub-key entirely from the basket key if the basket doesn't contain any fruits. This might need to be duplicated elsewhere if other keys rely on the fruits key as well.

[Jasha10]

Interesting. Yes, this is indeed an inconsistency. Thanks for bringing it up :)

[Jasha10]

You can work around this inconsistency by creating an empty file in the fruits group and including it in your defaults list:

# conf/config.yaml
defaults:
  - fruits/emptyfruit
  - fruits:
      - banana

Here's the empty file:

# conf/fruits/emptyfruit.yaml

At the command line:

$ python main.py 'fruits=[]'
0