Galileo-Galilei / kedro-mlflow

A kedro-plugin for integration of mlflow capabilities inside kedro projects (especially machine learning model versioning and packaging)
https://kedro-mlflow.readthedocs.io/
Apache License 2.0
195 stars 29 forks source link

Setup mlflow configuration when context is loaded #124

Closed Galileo-Galilei closed 2 years ago

Galileo-Galilei commented 3 years ago

Description

When I call load_context() interactively, the configuration of the mlflow.yml is:

Context

I always struggle when i want to use the configuration in the mlflow.yml interactively. This makes experimentation & debugging harder.

Possible Implementation

For now, I can't see how it is possible to modify the context object to add attributes. We can eventually add some to the catalog with the after_catalog_created hook, but the configuration should not be tied to the catalog execution.

takikadiri commented 3 years ago

I understand the point, but to my knowledge the kedro context do not manage plugins properties, it just use them to supercharge his own properties (catalog, params, pipelines)

It isn't sufficient for you to get the config by calling the get_mlflow_config of kedro_mlflow ?

context = load_context(Path.cwd())
mlflow_config = get_mlflow_config(context)
Galileo-Galilei commented 2 years ago

The current implementation seems indeed satisfying enough (with kedro>=0.17 and the a KedroSession activated, it can be simplified to

from kedro_mlflow.framework.context
mlflow_config=get_mlflow_config()
mlflow_config.setup()

However, we should setup automatically the configuration for jupyter, through the %reload_kedro line magic.