facebookresearch / hydra

Hydra is a framework for elegantly configuring complex applications
https://hydra.cc
MIT License
8.84k stars 637 forks source link

Overriding values of non-default config groups (without overriding the group itself) #2982

Open brfi3983 opened 1 week ago

brfi3983 commented 1 week ago

I am wondering if it is possible to override a certain config value in a config group that by default is not part of the defaults list.

For instance, imagine we have the following defaults list:

defaults:
  modules:
     - thk
     - iceflow

where each of the following groups have many parameters within (e.g. below)

grid_size: 100
units: meters
shape: [2,100,10]
etc...

Then, if I have an experiment called custom.yaml and I want to override just the units, I would like to specify it like this

@package __global__
modules:
  thk:
    units: centimeters

However, this only works if thk is part of the original defaults list. If I do not specify it as part of the list, then it will replace the entire thk module with just the units variable, where grid_size, 'shape' and the others will be lost. This makes sense to me and does not seem to be an issue. However, I am wondering what the best to go about this as this does pose an inconvenience for me.

Ideally, I would like to start with an empty defaults list, and if I specify a parameter within some config group (like thk: units: centimeters), it would then load the thk group so that the other parameters are also loaded. I could simply include all of these modules as part of the defaults: modules but then if I only use one of those modules like thk I do not want to have 100 other modules like iceflow that I never used. This way if I share my configuration with another person, they do not have a list of a bunch of random config groups they will not use.

So here is the question: is there a best way to go about this or does not not exist anything yet? I would appreciate any suggestions on how to make this possible. If it was not clear, let me know and I can clarify what I meant.