[FEATURE]: Simplifying your configs using hydra-zen

[rsokl]

Hello! I just came across your project - it looks really nice!

I noticed that you are using Hydra's yamls to make your experiments configurable. I figured I might put hydra-zen on your radar.

hydra-zen provides simple functions for dynamically creating dataclass-based configs (a.k.a structured configs), and it is especially useful for large-scale projects like yours.

There are a few big "wins" to this:

• your configs are in Python, so you (and users!) can actually import them, inherit from them, etc. Plus you don't have to hand-write yaml files.
• your configs are dynamically generated, so if you change your library's API, then your configs will automatically reflect this.
• hydra-zen will validate configs against the signatures of the classes that it configures, so you can easily include config-validation in your test suite (which is really useful for a project with the size of disent).

For example, this is all you need to generate configs for encoder and decoder models:

import disent
from hydra_zen import builds

EncoderLinearConfig = builds(disent.model.ae.EncoderLinear, populate_full_signature=True)
DecoderLinearLinearConfig = builds(disent.model.ae.DecoderLinearLinear, populate_full_signature=True)

You could then use simple utility functions to inject globally-specified settings (e.g. x_shape='${dataset.meta.x_shape}') into instances of these configs. This really helps keep you from having to repeat yourself.

I hope this is somewhat helpful (or at least interesting). Just wanted to bring this to your attention - feel free to close this issue 😄

[nmichlo]

Hi there! Thank you for bringing this to my attention, hydra-zen looks interesting. I'll definitely keep this in mind for investigation as part of a future release.

Would definitely require a large re-write. I am not entirely sure if hydra-zen would support some of the features I need with regards to overriding and downstream projects. Will definitely need to investigate this too.

[rsokl]

I am not entirely sure if hydra-zen would support some of the features I need with regards to overriding and downstream projects. Will definitely need to investigate this too.

I'm happy to provide insights towards this end if you have any questions.