search

facebookresearch / recipes

Recipes are a standard, well supported set of blueprints for machine learning engineers to rapidly train models using the latest research techniques without significant engineering overhead.Specifically, recipes aims to provide- Consistent access to pre-trained SOTA models ready for production- Reference implementations for SOTA research reproducibility, and infrastructure to guarantee correctness, efficiency, and interoperability.
BSD 3-Clause "New" or "Revised" License
288 stars 36 forks source link

Leveraging hydra-zen for auto-generated and validated Hydra configs #7

Open rsokl opened 2 years ago

rsokl commented 2 years ago

Hello! Thanks for creating this repo -- it looks like it will be very useful. Hydra + PyTorch Lightning is definitely a killer one-two punch!

I wanted to bring your attention to hydra-zen, which is a library that adds functionality to Hydra and simplifies the process of writing configs. I think that this could be very handy for recipes.

Rather than hand-write YAML configs, you can auto-generate configs for objects using hydra_zen.builds. Furthermore, these config-generating functions provide strict runtime and static checking of the configs, making it trivial to validate all of recipes configs during nightly builds.

Another perk of leveraging hydra-zen is that it can make recipes less dependent on the specific directory layout of your directory structure; e.g. you could avoid having users be required to add a file specifically to torchrecipes/launcher/.

I could go on and on, but I'll leave it at that. I would be happy to provide more details, answer questions, etc. I hope that you find this to be useful 😄

Here are our docs: https://mit-ll-responsible-ai.github.io/hydra-zen/ Our code (our project is very well-tested!): https://github.com/mit-ll-responsible-ai/hydra-zen And a quick example of using hydra-zen + PyTorch lightning: https://mit-ll-responsible-ai.github.io/hydra-zen/how_to/pytorch_lightning.html

kandluis commented 2 years ago

Hi @rsokl! Thank so much for taking a look at this repo, and for the feedback you're providing! This is exactly the sort of input we're looking to get from the community as we co-develop our recipes vision with the PyTorch (and associated) communities!

@tangbinh, given you've been spending a bit of time looking into how we can further improve our integration with hydra and our config-system, if you haven't already, it might be worth taking into account hydra-zen and what that might look like if we take this approach.

From a quick glance at what @rsokl shared, it certainly appears to address at least a subset of the concerns we've heard from our Meta users regarding the existing recipes!

rsokl commented 2 years ago

Wonderful! I think it is likely that my colleagues and I hit a lot of the same issues that you are alluding to. Using hydra-zen ended up being quite transformational in terms of removing the repetitiveness and hotspots for mistakes that made Hydra taxing for us to use at-scale across PyTorch-based projects. Ultimately it makes using Hydra more Python-centric (and a bit more intuitive), rather than requiring users to effectively learn a new DSL based on yamls.

We are finally mature enough, both in terms the stability of our code and the quality of our docs, to start putting hydra-zen on peoples' radars (you are the first we reached out to!). That being said, we have been collaborating with a core-dev from omegaconf/Hydra for a while now, to help ensure we maintain broad compatibility and synergy with Hydra.