Input parameters for runtime-images and runtimes are not available

pacospace commented 3 years ago

Describe the issue Input parameters for runtime-images and runtimes are not available when spawning Elyra in JH.

I'm not sure why this behavior happens. I run the image [1] locally and everything works fine.

To Reproduce Steps to reproduce the behavior:

Go to https://jupyterhub-opf-jupyterhub.apps.zero.massopen.cloud/
Spawn image s2i-lab-elyra:v0.0.6

Screenshots or log output See: https://github.com/operate-first/support/issues/146

Expected behavior All input parameters are available. Workaround currently is to use elyra cli to add runtimes and runtime-images.

Deployment information Describe what you've deployed and how:

Elyra version: 2.0.1
Operating system: ubi 8
Installation source: [1]
Deployment type: Open Data Hub v1.0.5

References [1] https://quay.io/repository/thoth-station/s2i-lab-elyra?tab=tags

ptitzler commented 3 years ago

@pacospace Can you please retrieve the JupyterHub log file (from the pod named jupyterhub-<X>-<UUID>) and the notebook server log file (from the pod named jupyterhub-nb-<UUID>)? I deployed JupyterHub/Elyra using the v1.0.5 ODH manifest yesterday in my cloud environment and things seem to work as expected.

ptitzler commented 3 years ago

I'm unable to recreate the problem in the specified environment. The metadata editor displays correctly.

ptitzler commented 3 years ago

No explicit bug fix should be required because the issue should already be addressed in later releases. The workaround is to not use the command palette (<cmd/ctrl> ) to open the runtime configuration wizard. Instead, select the tab marked Runtimes from the sidebar. (If there's another one named Kubeflow Pipelines runtimes ignore it - that's the one that does not work.)

pacospace commented 3 years ago

No explicit bug fix should be required because the issue should already be addressed in later releases. The workaround is to not use the command palette (<cmd/ctrl> ) to open the runtime configuration wizard. Instead, select the tab marked Runtimes from the sidebar. (If there's another one named Kubeflow Pipelines runtimes ignore it - that's the one that does not work.)

thanks @ptitzler! and what about runtime images which is not available in the sidebar when starting Elyra?

Screenshot from 2021-03-24 17-10-57

ptitzler commented 3 years ago

and what about runtime images which is not available in the sidebar when starting Elyra?

See https://github.com/opendatahub-io/s2i-lab-elyra/issues/22.

Edit 1: @pacospace can you please clarify whether you were referring to the fact that no default runtime configuration is automatically created for the Kubeflow deployment, or the problem that the runtime images wizard isn't accessible? If it's the latter then my earlier response wasn't correct.

ptitzler commented 3 years ago

I did try a local installation of version 2.0.1 (from PyPI) and was unable to recreate the problems. This is somewhat concerning because this might indicate that there is something special about the container image in ODH that might cause later images to show similar behavior as well. We need to explicitly test this scenario before we roll out the new image version.

akchinSTC commented 3 years ago

@ptitzler - that issue is for the default runtime metadata instance not being generated. @pacospace - did you mean runtime metadata or runtime images? The runtime images configuration can be accessed using: https://elyra.readthedocs.io/en/latest/user_guide/runtime-image-conf.html#managing-runtime-images-with-the-user-interface

pacospace commented 3 years ago

@ptitzler - that issue is for the default runtime metadata instance not being generated. @pacospace - did you mean runtime metadata or runtime images? The runtime images configuration can be accessed using: https://elyra.readthedocs.io/en/latest/user_guide/runtime-image-conf.html#managing-runtime-images-with-the-user-interface

right, that is what we do usually, but the issue here https://github.com/operate-first/support/issues/146 is happening for runtimes and runtime-images when trying to add a new one from the UI:

Screenshot from 2021-03-24 20-12-47

ptitzler commented 3 years ago

@pacospace

I run the image [1] locally ...

What's the exact command you are running?

pacospace commented 3 years ago

@pacospace

I run the image [1] locally ...

What's the exact command you are running?

podman run -p 8888:8888 quay.io/thoth-station/s2i-lab-elyra:v0.0.6 jupyter lab --debug

ptitzler commented 3 years ago

Thank you! I confirmed that the problem is only re-creatable using that image

$ docker run -p 8888:8888 quay.io/thoth-station/s2i-lab-elyra:v0.0.6 jupyter lab --debug --ip=0.0.0.0 --port=8888

but it's not re-creatable using the official v2.0.1 Elyra container image

$ docker run -it -p 8888:8888 docker.io/elyra/elyra:2.0.1 jupyter lab

or a locally installed PyPI version.

ptitzler commented 3 years ago

I ran the installation verification commands in the container image

$ jupyter serverextension list
config dir: /opt/app-root/etc/jupyter
    jupyter_lsp  enabled 
    - Validating...
      jupyter_lsp 1.1.3 OK
    jupyter_resource_usage  enabled 
    - Validating...
      jupyter_resource_usage  OK
    jupyterlab  enabled 
    - Validating...
      jupyterlab 3.0.7 OK
    jupyterlab_requirements  enabled 
    - Validating...
      jupyterlab_requirements 0.4.7 OK
    nbdime  enabled 
    - Validating...
      nbdime 3.0.0.b1 OK
    jupyter_nbextensions_configurator  enabled 
    - Validating...
      jupyter_nbextensions_configurator 0.4.1 OK

In the following output note @elyra/metadata-extension v2.1.0-rc.1 enabled OK. This is definitely not the right extension for this version of Elyra!

$ jupyter labextension list
JupyterLab v3.0.7
/opt/app-root/share/jupyter/labextensions
        jupyterlab_requirements v0.4.7 enabled OK (python, jupyterlab-requirements)
        @krassowski/jupyterlab-lsp v3.3.1 enabled OK (python, jupyterlab_lsp)
        @jupyter-server/resource-usage v0.5.0 enabled OK (python, jupyter-resource-usage)
        @jupyterlab/git v0.30.0-beta.1 enabled OK (python, jupyterlab-git)

Other labextensions (built into JupyterLab)
   app dir: /opt/app-root/share/jupyter/lab
        @elyra/code-snippet-extension v2.0.1 enabled OK
        @elyra/metadata-extension v2.1.0-rc.1 enabled OK
        @elyra/pipeline-editor-extension v2.0.1 enabled OK
        @elyra/python-editor-extension v2.0.1 enabled OK
        @elyra/theme-extension v2.0.1 enabled OK
        nbdime-jupyterlab v2.1.0-beta.1 enabled OK

$ jupyter server extension list
Config dir: /opt/app-root/src/.jupyter

Config dir: /opt/app-root/etc/jupyter
    elyra enabled
    - Validating elyra...
      elyra 2.0.1 OK
    jupyter_resource_usage enabled
    - Validating jupyter_resource_usage...
      jupyter_resource_usage  OK
    jupyter_lsp enabled
    - Validating jupyter_lsp...
      jupyter_lsp 1.1.3 OK
    jupyterlab enabled
    - Validating jupyterlab...
      jupyterlab 3.0.7 OK
    jupyterlab_git enabled
    - Validating jupyterlab_git...
      jupyterlab_git 0.30.0b1 OK
    jupyterlab_requirements enabled
    - Validating jupyterlab_requirements...
      jupyterlab_requirements 0.4.7 OK
    nbclassic enabled
    - Validating nbclassic...
      nbclassic  OK
    nbdime enabled
    - Validating nbdime...
      nbdime 3.0.0.b1 OK

Config dir: /usr/local/etc/jupyter

ptitzler commented 3 years ago

@pacospace Since the problem only surfaces in the s2i image we'll need to discuss how that image was built. Can you please advise who we should reach out to so we can set up a walkthrough and q&a meeting?

pacospace commented 3 years ago

@pacospace Since the problem only surfaces in the s2i image we'll need to discuss how that image was built. Can you please advise who we should reach out to so we can set up a walkthrough and q&a meeting?

Thanks @ptitzler for looking into this.

In the call you should have:

@harshad16 from Thoth Team that created the automated pipelines for building images (e.g https://github.com/opendatahub-io/s2i-lab-elyra) @vpavlin from ODH team

harshad16 commented 3 years ago

Thanks, @pacospace for thoroughly tracking the issue. We would also revisit the build and see why the wrong extension version is installed on the image.

ptitzler commented 3 years ago

fwiw, this is the output we are getting if we run those validation commands in the official Elyra 2.0.1 container image we've published:

$ jupyter serverextension list
config dir: /opt/conda/etc/jupyter
    jupyter_resource_usage  enabled 
    - Validating...
      jupyter_resource_usage  OK
    jupyterlab  enabled 
    - Validating...
      jupyterlab 3.0.5 OK
    nbdime  enabled 
    - Validating...
      nbdime 3.0.0.b1 OK
    nbgitpuller  enabled 
    - Validating...
      nbgitpuller 0.9.0 OK

$ jupyter server extension list
Config dir: /home/jovyan/.jupyter

Config dir: /opt/conda/etc/jupyter
    elyra enabled
    - Validating elyra...
      elyra 2.0.1 OK
    jupyter_lsp enabled
    - Validating jupyter_lsp...
      jupyter_lsp 1.1.1 OK
    jupyter_resource_usage enabled
    - Validating jupyter_resource_usage...
      jupyter_resource_usage  OK
    jupyterlab enabled
    - Validating jupyterlab...
      jupyterlab 3.0.5 OK
    jupyterlab_git enabled
    - Validating jupyterlab_git...
      jupyterlab_git 0.30.0b1 OK
    nbclassic enabled
    - Validating nbclassic...
      nbclassic  OK
    nbdime enabled
    - Validating nbdime...
      nbdime 3.0.0.b1 OK

Config dir: /usr/local/etc/jupyter

$ jupyter labextension list
JupyterLab v3.0.5
/opt/conda/share/jupyter/labextensions
        @jupyterlab/git v0.30.0-beta.1 enabled OK (python, jupyterlab-git)
        @krassowski/jupyterlab-lsp v3.2.0 enabled OK (python, jupyterlab_lsp)
        @jupyter-server/resource-usage v0.5.0 enabled OK (python, jupyter-resource-usage)

Other labextensions (built into JupyterLab)
   app dir: /opt/conda/share/jupyter/lab
        @elyra/code-snippet-extension v2.0.1 enabled OK
        @elyra/metadata-extension v2.0.1 enabled OK
        @elyra/pipeline-editor-extension v2.0.1 enabled OK
        @elyra/python-editor-extension v2.0.1 enabled OK
        @elyra/theme-extension v2.0.1 enabled OK
        nbdime-jupyterlab v2.1.0-beta.1 enabled OK

Disabled extensions:
    @jupyterlab/launcher-extension (all plugins)

akchinSTC commented 3 years ago

I suspect the reason we are seeing a 2.1.0 version of the extension is due this addition in the image build for v0.0.6 https://github.com/opendatahub-io/s2i-lab-elyra/commit/b4cfc4f826c12737207de192525ccd6358ef4eef

Elyra packages the front end extensions into our wheel so the user should not need to explicitly install the node packages themselves : https://github.com/elyra-ai/elyra/blob/2421627dab0565e5a72eb50032bfed75950e2b5f/setup.py#L130

ptitzler commented 3 years ago

This should be resolved in v0.0.7 and later. I checked the installed extension version numbers and they appear okay. I also wasn't able to reproduce the issue using the v0.0.7 image.

ptitzler commented 3 years ago

The v0.0.7 image is now available in ODH manifest repository v1.0.9: https://github.com/opendatahub-io/odh-manifests/tarball/v1.0.9

elyra-ai / elyra

Input parameters for runtime-images and runtimes are not available #1473