kedro mlflow ui: Error: No such command 'mlflow'

Galileo-Galilei / kedro-mlflow-tutorial

A tutorial on how to use kedro-mlflow plugin (https://github.com/Galileo-Galilei/kedro-mlflow) to synchronize training and inference and serve kedro pipeline

37 stars 5 forks source link

$ kedro mlflow ui Traceback (most recent call last): File "/home/chouffe/anaconda3/envs/kedro_mlflow_tutorial/lib/python3.7/site-packages/kedro/framework/cli/cli.py", line 683, in load_entry_points entry_point_commands.append(entry_point.load()) File "/home/chouffe/anaconda3/envs/kedro_mlflow_tutorial/lib/python3.7/site-packages/pkg_resources/__init__.py", line 2449, in load self.require(*args, **kwargs) File "/home/chouffe/anaconda3/envs/kedro_mlflow_tutorial/lib/python3.7/site-packages/pkg_resources/__init__.py", line 2472, in require items = working_set.resolve(reqs, env, installer, extras=self.extras) File "/home/chouffe/anaconda3/envs/kedro_mlflow_tutorial/lib/python3.7/site-packages/pkg_resources/__init__.py", line 777, in resolve raise VersionConflict(dist, req).with_context(dependent_req) pkg_resources.ContextualVersionConflict: (click 8.0.1 (/home/chouffe/anaconda3/envs/kedro_mlflow_tutorial/lib/python3.7/site-packages), Requirement.parse('click==7.*'), {'uvicorn'}) Error: Loading global commands from kedro-viz = kedro_viz.launchers.cli:commands Traceback (most recent call last): File "/home/chouffe/anaconda3/envs/kedro_mlflow_tutorial/lib/python3.7/site-packages/kedro/framework/cli/cli.py", line 683, in load_entry_points entry_point_commands.append(entry_point.load()) File "/home/chouffe/anaconda3/envs/kedro_mlflow_tutorial/lib/python3.7/site-packages/pkg_resources/__init__.py", line 2449, in load self.require(*args, **kwargs) File "/home/chouffe/anaconda3/envs/kedro_mlflow_tutorial/lib/python3.7/site-packages/pkg_resources/__init__.py", line 2472, in require items = working_set.resolve(reqs, env, installer, extras=self.extras) File "/home/chouffe/anaconda3/envs/kedro_mlflow_tutorial/lib/python3.7/site-packages/pkg_resources/__init__.py", line 777, in resolve raise VersionConflict(dist, req).with_context(dependent_req) pkg_resources.ContextualVersionConflict: (alembic 1.7.1 (/home/chouffe/anaconda3/envs/kedro_mlflow_tutorial/lib/python3.7/site-packages), Requirement.parse('alembic<=1.4.1'), {'mlflow'}) Error: Loading global commands from kedro_mlflow = kedro_mlflow.framework.cli.cli:commands Traceback (most recent call last): File "/home/chouffe/anaconda3/envs/kedro_mlflow_tutorial/lib/python3.7/site-packages/kedro/framework/cli/cli.py", line 683, in load_entry_points entry_point_commands.append(entry_point.load()) File "/home/chouffe/anaconda3/envs/kedro_mlflow_tutorial/lib/python3.7/site-packages/pkg_resources/__init__.py", line 2449, in load self.require(*args, **kwargs) File "/home/chouffe/anaconda3/envs/kedro_mlflow_tutorial/lib/python3.7/site-packages/pkg_resources/__init__.py", line 2472, in require items = working_set.resolve(reqs, env, installer, extras=self.extras) File "/home/chouffe/anaconda3/envs/kedro_mlflow_tutorial/lib/python3.7/site-packages/pkg_resources/__init__.py", line 777, in resolve raise VersionConflict(dist, req).with_context(dependent_req) pkg_resources.ContextualVersionConflict: (alembic 1.7.1 (/home/chouffe/anaconda3/envs/kedro_mlflow_tutorial/lib/python3.7/site-packages), Requirement.parse('alembic<=1.4.1'), {'mlflow'}) Error: Loading project commands from kedro_mlflow = kedro_mlflow.framework.cli.cli:commands Usage: kedro [OPTIONS] COMMAND [ARGS]... Try 'kedro -h' for help. Error: No such command 'mlflow'.

Hi,

glad to see that you are trying the plugin, and sorry to hear that you are facing so many issues.

I think the issues #6 #7 and this one have all the same cause: your environment seems completely messed up and it breaks dependencies import.

What is going on

Your stacktrace is showing a bunch of pkg_resources.ContextualVersionConflict :

with alembic (you have 1.7.1 and your mlflow version requires alembic<=1.4.1),
with click (you have click==8.0.1 while uvicorn (an mlflow dependency) requires click==7.*

These conflicts between the different versions of your package have likely raised some warning during their installation with pip and you ignored them. It may be ok most of the times, but kedro leverages setuptools entrypoints to add hooks and CLI command from plugins to their existing CLI, and setuptools is much "stricter" about dependencies, and it fails at runtime. These dependencies conflicts completely messes up the whole kedro CLI mangement system (including kedro ipython as you experienced in #7, but also very likely kedro viz)

Why did this happen

I can think of two cases in which such a situation arise:

you are using an old pip version (before october 2020) which has no dependency resolver. In this case, the install order of the package define which version of click will be installed , and if reinstalling a package with a new version beak an already installed dependency, it will just continue without any warning. This is a known bug in pip. To avoid this, either use a recent pip or install the package manually by hand in the requirements and check out if one install tries to override a package already installed by a previous one.
you have installed manually some package to get some new functionalities (say pip install --upgrade kedro-viz after installing the requirements.txt file of the project, and it updates some important packages (e.g. kedro, mlflow, click) with new dependencies in conflict with the existing ones.

How to solve it

Install a recent pip version and reinstall everything in a fresh conda environment to make it work:

conda create -n kmt python=3.7 -y
conda activate kmt
pip install --upgrade pip
pip install -e kedro-mlflow-tutorial/src # install the project as a package in editable mode, hence its src/requirements.txt

What to do if it does not work?

It seems that you did this in #7 and it solved your problem. Does this issue persist? If yes, can you give me:

the pip freeze of your environment (check that the version of kedro, mlflow and kedro-mlflow matches the one in the project requirements.txt file
your OS informations
the new stacktrace

Galileo-Galilei / kedro-mlflow-tutorial