search

kedro-org / kedro-devrel

Kedro developer relations team use this for content creation ideation and execution
Apache License 2.0
0 stars 3 forks source link

Create Kedro Academy materials about configuration files #19

Closed stichbury closed 1 year ago

stichbury commented 1 year ago

This is a child task of kedro-org/kedro-devrel#13 and covers creation of MVP slides about configuration files for Kedro Academy.

stichbury commented 1 year ago

I've left a v1 of this file in Box in the usual folder for draft decks

astrojuanlu commented 1 year ago

Done giving this a first pass. Been collecting some thoughts on alternative configuration systems, I was not aware of OmegaConf, Hydra, and anyconfig. Looks like Hydra extends OmegaConf and is used by lots of Facebook projects as well as DVC. anyconfig is mostly used by Kedro as far as I could see, and in fact it was decided to migrate to OmegaConf. Therefore, I'd propose narrowing down this part and focusing on OmegaConf exclusively, and maybe mentioning Hydra as a more powerful system built on top of it.

About hyperparameter tuning, seems like Hydra is often used in conjunction with something else, like Optuna https://medium.com/optuna/easy-hyperparameter-management-with-hydra-mlflow-and-optuna-783730700e7d. Worth mentioning as well with proper context.

About data access, I like how both Intake and Kedro define YAML-based data catalogs, each one with its own convention. Again, worth mentioning.

Finally, Kedro kind of supports all those use cases, so it's worth pointing out as well.

@stichbury @yetudada wondering if you want to complete that pending slide, or do you prefer me to do it?

Apart from that, I'll take care of the exercise.

astrojuanlu commented 1 year ago

This can be closed.