search

thoth-station / jupyterlab-requirements

JupyterLab Extension for dependency management and optimization
GNU General Public License v3.0
16 stars 9 forks source link

Can't change one of the dependency packages #269

Closed codificat closed 3 years ago

codificat commented 3 years ago

Describe the bug

When I went to "Manage Dependencies" in my notebook, the list of automatically detected packages had one that needed a change: the dependency is google-cloud-storage but it was showing up as google.

Trying to fix that package name does not seem to work

To Reproduce

Steps to reproduce the behavior: on MOC:

  1. Launch s2i-lab-elyra:v0.0.7 (Default size, no GPUs, no env)
  2. Use the git extension to clone https://github.com/aicoe-aiops/ocp-ci-analysis.git
  3. Open the notebook: ocp-ci-analysis/notebooks/data-sources/gcsweb-ci/build-logs/build_log_EDA.ipynb
  4. Click on "Manage Dependencies"
  5. One of the dependencies listed is google. Click the "edit" icon for that.
    • package name goes blank and to the bottom of the list
    • fill in the blank package name with google-cloud-storage
    • click on the "save dependencies" icon
  6. The list of package dependencies still lists google instead of google-cloud-storage

Expected behavior

I would expect to be able to change the name of that dependency.

Screenshots If applicable, add other screenshots to help explain your problem: e.g. outputs from https://github.com/thoth-station/jupyterlab-requirements#troubleshoot

$ jupyter server extension list
Config dir: /opt/app-root/src/.jupyter

/opt/app-root/lib64/python3.8/site-packages/jupyter_server/transutils.py:13: FutureWarning: The alias `_()` will be deprecated. Use `_i18n()` instead.
  warnings.warn(warn_msg, FutureWarning)
/opt/app-root/lib64/python3.8/site-packages/jupyter_server/transutils.py:13: FutureWarning: The alias `_()` will be deprecated. Use `_i18n()` instead.
  warnings.warn(warn_msg, FutureWarning)
Config dir: /opt/app-root/etc/jupyter
    elyra enabled
    - Validating elyra...
      elyra 2.2.1 OK
    jupyter_lsp enabled
    - Validating jupyter_lsp...
      jupyter_lsp 1.1.4 OK
    jupyter_resource_usage enabled
    - Validating jupyter_resource_usage...
      jupyter_resource_usage  OK
    jupyterlab enabled
    - Validating jupyterlab...
      jupyterlab 3.0.12 OK
    jupyterlab_git enabled
    - Validating jupyterlab_git...
      jupyterlab_git 0.30.0b2 OK
    jupyterlab_requirements enabled
    - Validating jupyterlab_requirements...
      jupyterlab_requirements 0.6.3 OK
    nbclassic enabled
    - Validating nbclassic...
      nbclassic  OK
    nbdime enabled
    - Validating nbdime...
      nbdime 3.0.0.b1 OK

Config dir: /usr/local/etc/jupyter

$ jupyter labextension list
/opt/app-root/lib64/python3.8/site-packages/jupyter_server/transutils.py:13: FutureWarning: The alias `_()` will be deprecated. Use `_i18n()` instead.
  warnings.warn(warn_msg, FutureWarning)
JupyterLab v3.0.12
/opt/app-root/share/jupyter/labextensions
        jupyterlab_requirements v0.6.3 enabled OK (python, jupyterlab-requirements)
        @jupyter-server/resource-usage v0.5.0 enabled OK (python, jupyter-resource-usage)
        @jupyterlab/git v0.30.0-beta.2 enabled OK (python, jupyterlab-git)
        @krassowski/jupyterlab-lsp v3.5.0 enabled OK (python, jupyterlab_lsp)

Other labextensions (built into JupyterLab)
   app dir: /opt/app-root/share/jupyter/lab
        @elyra/code-snippet-extension v2.2.1 enabled OK
        @elyra/metadata-extension v2.2.1 enabled OK
        @elyra/pipeline-editor-extension v2.2.1 enabled OK
        @elyra/python-editor-extension v2.2.1 enabled OK
        @elyra/r-editor-extension v2.2.1 enabled OK
        @elyra/theme-extension v2.2.1 enabled OK
        nbdime-jupyterlab v2.1.0-beta.1 enabled OK

Disabled extensions:
    @jupyterlab/launcher-extension (all plugins)

$ jupyter kernelspec list
Available kernels:
  jupyterlab-requirements    /opt/app-root/src/.local/share/jupyter/kernels/jupyterlab-requirements
  python3                    /opt/app-root/share/jupyter/kernels/python3

Environment information Describe the environment from where you are using the extension:

Stack information If you have issue for resolving your stack with thoth resolution engine. Please state the dependencies in your stack below.

I believe that the issue reported here is prior to dep resolution, but just in case the notebook imports:

import json
import matplotlib.pyplot as plt
import pandas as pd
import os
import seaborn as sns
from google.cloud import storage

and the git repo lists a few packages in its Pipfile

Additional context Add any other context about the problem here.

pacospace commented 3 years ago

Describe the bug

When I went to "Manage Dependencies" in my notebook, the list of automatically detected packages had one that needed a change: the dependency is google-cloud-storage but it was showing up as google.

Trying to fix that package name does not seem to work

To Reproduce

Steps to reproduce the behavior: on MOC:

  1. Launch s2i-lab-elyra:v0.0.7 (Default size, no GPUs, no env)
  2. Use the git extension to clone https://github.com/aicoe-aiops/ocp-ci-analysis.git
  3. Open the notebook: ocp-ci-analysis/notebooks/data-sources/gcsweb-ci/build-logs/build_log_EDA.ipynb
  4. Click on "Manage Dependencies"

  5. One of the dependencies listed is google. Click the "edit" icon for that.

    • package name goes blank and to the bottom of the list
    • fill in the blank package name with google-cloud-storage
    • click on the "save dependencies" icon
  6. The list of package dependencies still lists google instead of google-cloud-storage

Expected behavior

I would expect to be able to change the name of that dependency.

Screenshots If applicable, add other screenshots to help explain your problem: e.g. outputs from https://github.com/thoth-station/jupyterlab-requirements#troubleshoot

$ jupyter server extension list
Config dir: /opt/app-root/src/.jupyter

/opt/app-root/lib64/python3.8/site-packages/jupyter_server/transutils.py:13: FutureWarning: The alias `_()` will be deprecated. Use `_i18n()` instead.
  warnings.warn(warn_msg, FutureWarning)
/opt/app-root/lib64/python3.8/site-packages/jupyter_server/transutils.py:13: FutureWarning: The alias `_()` will be deprecated. Use `_i18n()` instead.
  warnings.warn(warn_msg, FutureWarning)
Config dir: /opt/app-root/etc/jupyter
    elyra enabled
    - Validating elyra...
      elyra 2.2.1 OK
    jupyter_lsp enabled
    - Validating jupyter_lsp...
      jupyter_lsp 1.1.4 OK
    jupyter_resource_usage enabled
    - Validating jupyter_resource_usage...
      jupyter_resource_usage  OK
    jupyterlab enabled
    - Validating jupyterlab...
      jupyterlab 3.0.12 OK
    jupyterlab_git enabled
    - Validating jupyterlab_git...
      jupyterlab_git 0.30.0b2 OK
    jupyterlab_requirements enabled
    - Validating jupyterlab_requirements...
      jupyterlab_requirements 0.6.3 OK
    nbclassic enabled
    - Validating nbclassic...
      nbclassic  OK
    nbdime enabled
    - Validating nbdime...
      nbdime 3.0.0.b1 OK

Config dir: /usr/local/etc/jupyter

$ jupyter labextension list
/opt/app-root/lib64/python3.8/site-packages/jupyter_server/transutils.py:13: FutureWarning: The alias `_()` will be deprecated. Use `_i18n()` instead.
  warnings.warn(warn_msg, FutureWarning)
JupyterLab v3.0.12
/opt/app-root/share/jupyter/labextensions
        jupyterlab_requirements v0.6.3 enabled OK (python, jupyterlab-requirements)
        @jupyter-server/resource-usage v0.5.0 enabled OK (python, jupyter-resource-usage)
        @jupyterlab/git v0.30.0-beta.2 enabled OK (python, jupyterlab-git)
        @krassowski/jupyterlab-lsp v3.5.0 enabled OK (python, jupyterlab_lsp)

Other labextensions (built into JupyterLab)
   app dir: /opt/app-root/share/jupyter/lab
        @elyra/code-snippet-extension v2.2.1 enabled OK
        @elyra/metadata-extension v2.2.1 enabled OK
        @elyra/pipeline-editor-extension v2.2.1 enabled OK
        @elyra/python-editor-extension v2.2.1 enabled OK
        @elyra/r-editor-extension v2.2.1 enabled OK
        @elyra/theme-extension v2.2.1 enabled OK
        nbdime-jupyterlab v2.1.0-beta.1 enabled OK

Disabled extensions:
    @jupyterlab/launcher-extension (all plugins)

$ jupyter kernelspec list
Available kernels:
  jupyterlab-requirements    /opt/app-root/src/.local/share/jupyter/kernels/jupyterlab-requirements
  python3                    /opt/app-root/share/jupyter/kernels/python3

Environment information Describe the environment from where you are using the extension:

  • jupyterlab version: v3.0.12
  • jupyterlab-requirements version: v0.6.3
  • Installation source: built into the s2i-lab-elyra:v0.0.7 image
  • Operating system: RHEL 8.3
  • Environment type: JupyterHub spawned image
  • [Optional] Add URL to pull image quay.io/thoth-station/s2i-lab-elyra@sha256:99ccb1f7b1488ae2086fe378cad3d04427d71cdcd5cc25b1358d3c12cfdbf739

Stack information If you have issue for resolving your stack with thoth resolution engine. Please state the dependencies in your stack below.

I believe that the issue reported here is prior to dep resolution, but just in case the notebook imports:

import json
import matplotlib.pyplot as plt
import pandas as pd
import os
import seaborn as sns
from google.cloud import storage

and the git repo lists a few packages in its Pipfile

I believe not all of them are used in the same notebook. The use of overlays should help split that huge stack: https://github.com/aicoe-aiops/ocp-ci-analysis/issues/261.

Additional context Add any other context about the problem here.

Thanks a lot for opening this issue @codificat! the image you are using contains juyterlab-requirements in version 0.6.3, I would check first of all if we have similar behaviour with the new elyra image v0.0.8, it should be the experimental one listed in JH, containing jupyterlab-requiremens 0.6.5 which solved different bugs. Meanwhile I will investigate improvements to make more robust behaviour. Thank you very much! Feel free to ask all questions here :)

codificat commented 3 years ago

Thanks a lot for opening this issue @codificat! the image you are using contains juyterlab-requirements in version 0.6.3, I would check first of all if we have similar behaviour with the new elyra image v0.0.8, it should be the experimental one listed in JH, containing jupyterlab-requiremens 0.6.5 which solved different bugs. Meanwhile I will investigate improvements to make more robust behaviour. Thank you very much! Feel free to ask all questions here :)

Thanks! I went to try this quickly - at first sight it seems the problem is still there with the experimental v0.0.8 image... but I only ran a quick test - wondering if something was cached on my pv maybe? I am getting 504 Gateway timeouts when I save, maybe not a good time to test now, will try again later.

pacospace commented 3 years ago

Thanks a lot for opening this issue @codificat! the image you are using contains juyterlab-requirements in version 0.6.3, I would check first of all if we have similar behaviour with the new elyra image v0.0.8, it should be the experimental one listed in JH, containing jupyterlab-requiremens 0.6.5 which solved different bugs. Meanwhile I will investigate improvements to make more robust behaviour. Thank you very much! Feel free to ask all questions here :)

Thanks! I went to try this quickly - at first sight it seems the problem is still there with the experimental v0.0.8 image... but I only ran a quick test - wondering if something was cached on my pv maybe? I am getting 504 Gateway timeouts when I save, maybe not a good time to test now, will try again later.

I'm working on a solution to improve the experience on these 504 gateway timeout and to provide more info on issues to the user. Thanks @codificat!

codificat commented 3 years ago

I just gave another quick try to this, with the image:

$ echo $JUPYTER_IMAGE
experimental-s2i-lab-elyra:v0.0.8

I had also moved away a few dirs from $HOME in the PVC, including .jupyter and .local (side comment: at first sight I could not find a way to just delete the PVC itself to start from scratch... it seems I don't have perms for that).

That image seems to have version 0.0.64, not 0.0.65:

$ jupyter labextension list
JupyterLab v3.0.14
/opt/app-root/share/jupyter/labextensions
        jupyterlab_requirements v0.6.4 enabled OK (python, jupyterlab-requirements)
        nbdime-jupyterlab v2.1.0 enabled OK
        @jupyterlab/git v0.30.0 enabled OK (python, jupyterlab-git)
        @jupyter-server/resource-usage v0.5.0 enabled OK (python, jupyter-resource-usage)
        @krassowski/jupyterlab-lsp v3.6.0 enabled OK (python, jupyterlab-lsp)

Other labextensions (built into JupyterLab)
   app dir: /opt/app-root/share/jupyter/lab
        @elyra/code-snippet-extension v2.2.4 enabled OK
        @elyra/metadata-extension v2.2.4 enabled OK
        @elyra/pipeline-editor-extension v2.2.4 enabled OK
        @elyra/python-editor-extension v2.2.4 enabled OK
        @elyra/r-editor-extension v2.2.4 enabled OK
        @elyra/theme-extension v2.2.4 enabled OK

Disabled extensions:
    @jupyterlab/launcher-extension (all plugins)

with that, the problem editing the dependency was still there: it would save google instead of google-cloud-storage after trying to edit that name.

Maybe I need to try a newer version...

pacospace commented 3 years ago

I just gave another quick try to this, with the image:

$ echo $JUPYTER_IMAGE
experimental-s2i-lab-elyra:v0.0.8

I had also moved away a few dirs from $HOME in the PVC, including .jupyter and .local (side comment: at first sight I could not find a way to just delete the PVC itself to start from scratch... it seems I don't have perms for that).

This might be a use case for some users, there should be doc in operate first on how to do that or at least on how to ask for deletion of PVC, please open an issue in https://github.com/operate-first/support to request doc (maybe I could not find them).

That image seems to have version 0.0.64, not 0.0.65:

$ jupyter labextension list
JupyterLab v3.0.14
/opt/app-root/share/jupyter/labextensions
        jupyterlab_requirements v0.6.4 enabled OK (python, jupyterlab-requirements)
        nbdime-jupyterlab v2.1.0 enabled OK
        @jupyterlab/git v0.30.0 enabled OK (python, jupyterlab-git)
        @jupyter-server/resource-usage v0.5.0 enabled OK (python, jupyter-resource-usage)
        @krassowski/jupyterlab-lsp v3.6.0 enabled OK (python, jupyterlab-lsp)

Other labextensions (built into JupyterLab)
   app dir: /opt/app-root/share/jupyter/lab
        @elyra/code-snippet-extension v2.2.4 enabled OK
        @elyra/metadata-extension v2.2.4 enabled OK
        @elyra/pipeline-editor-extension v2.2.4 enabled OK
        @elyra/python-editor-extension v2.2.4 enabled OK
        @elyra/r-editor-extension v2.2.4 enabled OK
        @elyra/theme-extension v2.2.4 enabled OK

Disabled extensions:
    @jupyterlab/launcher-extension (all plugins)

with that, the problem editing the dependency was still there: it would save google instead of google-cloud-storage after trying to edit that name.

Maybe I need to try a newer version...

I'm working on a new release to solve these issues with UI.

pacospace commented 3 years ago

@codificat sorry it took some time due to other work. I was able to replicate the issue and introduced the solution already in https://github.com/thoth-station/jupyterlab-requirements/pull/286. Next release will contain the fix for this!

codificat commented 3 years ago

Great, thanks!