adding yaml config interpolation

[stas00]

Is your feature request related to a problem? Please describe.

Could SimpleParsing add support for interpolation? here is an example of how omegaconf does it: https://omegaconf.readthedocs.io/en/2.3_branch/usage.html#config-node-interpolation

Describe the solution you'd like

So that we could do something like this:

Input YAML file:

server:
  host: localhost
  port: 80

client:
  url: http://${server.host}:${server.port}/
  server_port: ${server.port}
  # relative interpolation
  description: Client of ${.url}

Thank you!

[lebrice]

Hello there @stas00 , thanks for posting! :)

I'm familiar with Hydra/OmegaConf, and this feature is very powerful. I agree that it would be great to also have this in Simple-Parsing. One way to do this could be to add omegaconf as an optional extra dependency, and reuse as much of their interpolation code as possible, with some custom logic as needed.

I'm assuming we'd want to use this only when parsing config files, correct? Otherwise I could also imagine a nice Python api with something like an interpolated_field variant of the dataclasses.field function:

from dataclasses import dataclass
from simple_parsing import interpolated_field, parse

@dataclass
class Server:
    host: str
    port: int

@dataclass
class Client
    url: str = interpolated_field("https://${server.host}:${server.port}/")
    server_port: int = interpolated_field("${server.port}")
    #relative interpolation
    description: str = interpolated_field("Client of ${.url}")

@dataclass
class Config:
    server: Server
    client: Client

config = parse(Config)

Note: For the last kind of (relative) interpolation, I made this little gist for what I call conditional_field, which you might find interesting: https://gist.github.com/lebrice/728f0e0218dcef6f74cbebd70af5857e#file-conditional_fields-py-L145-L160 (You can also directly pip-install that gist if you want to try it out.)

I think that adding interpolation in just yaml files first would be a (challenging) good start. It shouldn't be much more work to later add a helper function to also make this possible in the dataclasses, if we think this is useful.

Thanks again for posting, let me know what you think! :)

[stas00]

Hi Fabrice,

I agree that starting with interpolation support in just the yaml file would be great!

For the dataclasses python code one could already use the __post_init__ to interpolate the data at loading time. But, of course, the solution you proposed looks better.

[lebrice]

__post_init__ doesn't easily let you interpolate with fields outside your own dataclass instance though, except if you're using some kind of global variable or contextvar, I suppose.

[stas00]

Indeed, the top dataclass class will receive access to all parsed configs, so that's where we currently manipulate things.

[jerpint]

+1 to adding suport for Omega-conf style interpolation!