kedro-mlflow CLI is unavailable inside a Kedro project

dmb23 commented 3 years ago

Description

I try to reproduce the minimal example from the Docs: a Kedro project using the starter pandas-iris using the kedro-mlflow functinality. I do not arrive at initializing the kedro-mlflow project, since the cli commands are not available.

Context

It is unclear to me if this is connected to #157 I wanted to start looking into kedro-mlflow, but got immediatle blocked by the initialization of the project. Therefore any advice on where to look to fix this would also be appreciated.

Steps to Reproduce

conda create -n kedro_mlflow python=3.8
conda activate kedro_mlflow
pip install kedro-mlflow
kedro mlflow -h
kedro new --starter=pandas-iris
cd mlflow_test/
kedro mlflow -h
> ERROR "No such command 'mlflow'"

Expected Result

kedro mlflow is available in a project directory, i.e. kedro mlflow -h gives the same output inside the folder as before

Actual Result

inside the project folder the mlflow command is unknown to Kedro

.../miniconda3/envs/kedro_mlflow/lib/python3.8/site-packages/pkg_resources/__init__.py:1130: DeprecationWarning: Use of .. or absolute path in a resource path is not allowed and will raise exceptions in a future release.
  return get_provider(package_or_requirement).get_resource_filename(
..../miniconda3/envs/kedro_mlflow/lib/python3.8/site-packages/mlflow/types/schema.py:49: DeprecationWarning: `np.object` is a deprecated alias for the builtin `object`. To silence this warning, use `object` by itself. Doing this will not modify any behavior and is safe. 
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
  binary = (7, np.dtype("bytes"), "BinaryType", np.object)
2021-04-23 17:49:52,197 - root - INFO - Registered hooks from 2 installed plugin(s): kedro-mlflow-0.7.1
Usage: kedro [OPTIONS] COMMAND [ARGS]...
Try 'kedro -h' for help.

Error: No such command 'mlflow'.

Your Environment

Ubuntu 18.04.5

Kedro 0.17.3
kedro-mlflow 0.7.1
python 3.8.8.
mlflow 1.15.0

Does the bug also happen with the last version on master?

yes

Galileo-Galilei commented 3 years ago

Hi,

I wil try to check it out this weekend, but the kedro==0.17.3 version is brand new (it was released yesterday), and given my experience with past kedro versions update 2 things might have happened on kedro's side:

They have broken the auto-discovery mechanism (I've seen in the release note that they change the CLI command discovery to enale overriding project commands by plugins)
They have not updated their pandas-iris starter yet which does not match the new version and is only compliant with kedro==0.17.2.

While I am investigating, would you please confirm that :

kedro-mlflow works fine with kedro==0.17.2 with your setup
kedro-mlflow works fine if you don't use the pandas-iris starter: try kedro new with kedro==0.17.3 and then add one ode to test the plugin
I'd be glad to see if another plugin (e.g. kedro-viz) is facing the same problem that kedro-mlflow. Would you mind checking?

Of course there is the possibility that the problem comes from kedro-mlflow itself, but I hardly believe it. I'll tell you within 2 days.

Galileo-Galilei commented 3 years ago

I am sorry, I am quite busy for now and I will not debug this before next week. Once again, it is very likely kedro's plugin discovery mechanism has been broken in the new release, I strongly suggest you go back to kedro==0.17.2.

Next actions:

[X] reproduce the bug -> Done, thanks for the very good reproducible example
[X] Check if it happens with other plugins (say kedro-viz) -> kedro viz global command is properly discovered
[X] Check if hooks are properly loaded -> everything works fine if I add a mlflow.yml manually in the conf/local folder (or any folder in conf/ actually). -> This is a short term solution for you,e ven if it is not very convenient. You can find allowed keys in the documentation or irectly copy paste it from the code
[X] Check if the tests pass with kedro==0.17.3 -> Some tests are failing, but not the one related to the CLI commands which seems discovered. I need to investigate further.
[x] Check if other plugins with local commands are discovered
[x] Check if it also happens it an empty project (i.e. not a starter)

dmb23 commented 3 years ago

First of all, thank you for looking so quickly into it!

From how I read your second message you already know that, but to answer your questions:

detecting kedro mlflow works fine with kedro==0.17.2
the problem is consistent with kedro==0.17.3 independent if I use the pandas-iris starter or not
kedro viz is found also with kedro=0.17.3

Again, thank you for providing workarounds directly on Monday morning, I can nicely work with those! A question for my understanding of the plugin: As long as the hooks are loaded, the mlflow functionality depends only on a mlflow.yml to be present, and all that kedro mlflow init does is copy this file from the template into conf/local, is this correct?

Galileo-Galilei commented 3 years ago

TL;DR:

Install this version for now, it should make the command available again:

pip uninstall kedro-mlflow
pip install git+https://github.com/Galileo-Galilei/kedro-mlflow.git@bug/no-cli

Beware: it is very important to uninstall your existing version of kedro-mlflow before reinstalling because the patch has the same version number that the current release.

If you confirm this works for you, I will deploy the patch to PyPI before kedro provides a patch on their side.

Hi, some follow-up about this bug:

I've figured out what is going on but not why it happens. The mlflow group of command exists both at global (new) and project (init, ui) levels and for an unknown reason, kedro takes into account only one group of command in its 0.17.3 version. This is a bug I will report to the core team. However, it does not affect their other plugins (kedro-viz, kedro-docker, kedro-airflow) because none of them has both global and project commands.
The quickest (hacky) fix is to remove the global group of command to the make the project ones available. I've done this in the branch bug/no-cli of the repo.

To answer your question:

A question for my understanding of the plugin: As long as the hooks are loaded, the mlflow functionality depends only on a mlflow.yml to be present, and all that kedro mlflow init does is copy this file from the template into conf/local, is this correct?

Exactly: the init command renders the template (i.e. copy paste it + replace the jinja tags with dynamic values like the name of your project) to a folder in your conf/ folder (by default local, but you can specify an environment like this: kedro mlflow init --env=<your-env-folder>). The hooks contain all the code logic and this mlflow.yml file is just here to pass parameters to them.

The other project command is kedro mlflow ui which is just a wrapper of "mlflow ui" with the parameters (mlflow_tracking_uri, port, host) defined in your mlflow.yml file.

dmb23 commented 3 years ago

thanks, form a quick test I would say: the patch works like a charm!

Galileo-Galilei commented 3 years ago

Hi @dmb23, I've just deployed the patch to PyPI. You can use `pip install kedro_mlflow==0.7.2`` and it should be ok for now. I close the issue, but feel free to reopen if you still encounter any issue in this new version.

Galileo-Galilei / kedro-mlflow