getindata / kedro-kubeflow

Kedro Plugin to support running workflows on Kubeflow Pipelines
https://kedro-kubeflow.readthedocs.io
Apache License 2.0
46 stars 21 forks source link

Provide a docker image with JupyterLab environment to use with AI Platform Notebooks #30

Closed em-pe closed 3 years ago

em-pe commented 3 years ago

Notebook images provided by Google cause dependency conflicts when used with both kedro and kedro-kubeflow plugin. We could provide a docker image with JupyterLab and packages compatible with current plugin version, so it's easy to use for new kedro projects.

szczeles commented 3 years ago

It looks it works if you create virtualenv first:

(venv-demo) (base) jupyter@python-mariusz:~/kubeflow-plugin-demo$ kedro install
No requirements.in found. Copying contents from requirements.txt...
/home/jupyter/venv-demo/bin/python -m piptools compile -q /home/jupyter/kubeflow-plugin-demo/src/requirements.in
/home/jupyter/venv-demo/bin/python -m pip install -U -r /home/jupyter/kubeflow-plugin-demo/src/requirements.txt
...
Successfully installed appdirs-1.4.4 argon2-cffi-20.1.0 async-generator-1.10 backcall-0.2.0 black-19.10b0 bleach-3.2.1 coverage-5.3.1 decorator-4.4.2 defusedxml-0.6.0 entrypoints-0.3 flake8-3.8.4 flask-1.1.2 ipykernel-5.4.3 ipython-7.19.0 ipywidgets-7.6.3 isort-4.3.21 itsdangerous-1.1.0 jedi-0.18.0 joblib-1.0.0 jupyter-1.0.0 jupyter-console-6.2.0 jupyterlab-0.31.1 jupyterlab-launcher-0.10.5 jupyterlab-pygments-0.1.2 jupyterlab-widgets-1.0.0 kedro-viz-3.8.1 mccabe-0.6.1 mistune-0.8.4 more-itertools-8.6.0 nbclient-0.5.1 nbconvert-6.0.7 nbformat-5.0.8 nbstripout-0.3.3 nest-asyncio-1.4.3 notebook-6.1.6 numpy-1.19.5 packaging-20.8 pandas-1.2.0 pandocfilters-1.4.3 parso-0.8.1 pathspec-0.8.1 pexpect-4.8.0 pickleshare-0.7.5 prometheus-client-0.9.0 prompt-toolkit-3.0.10 ptyprocess-0.7.0 py-1.10.0 pycodestyle-2.6.0 pyflakes-2.2.0 pygments-2.7.4 pyparsing-2.4.7 pytest-5.4.3 pytest-cov-2.10.1 pytest-mock-1.13.0 qtconsole-5.0.1 qtpy-1.9.0 regex-2020.11.13 scikit-learn-0.23.2 scipy-1.6.0 send2trash-1.5.0 terminado-0.9.2 testpath-0.4.4 threadpoolctl-2.1.0 toml-0.10.2 typed-ast-1.4.2 wcwidth-0.2.5 webencodings-0.5.1 werkzeug-1.0.1 wheel-0.32.2 widgetsnbextension-3.5.1 xlrd-1.2.0 xlsxwriter-1.3.7
WARNING: You are using pip version 20.3.1; however, version 20.3.3 is available.
You should consider upgrading via the '/home/jupyter/venv-demo/bin/python -m pip install --upgrade pip' command.
Requirements installed!