search

microsoft / vscode-jupyter-hub

Jupyter Hub Support in VS Code
https://marketplace.visualstudio.com/items?itemName=ms-toolsai.jupyter-hub
MIT License
8 stars 6 forks source link

Jupyterhub kernel access with extensions for users with certain special characters in their username fails to access extension.js code from Jupyterhub #55

Closed quantum-byte closed 4 months ago

quantum-byte commented 4 months ago

Environment data

Expected behaviour

When using a Jupyterhub instance with Jupyterlab extensions setup (In my example this will be plotely) i can use those extensions also locally while executing a local Jupyter Notebook in VS code with the remote kernel on the Jupyterhub. Since the executed python code is from Jupyterhub also the extension js code should be downloaded from the Jupyterhub instance for compatibility reasons.

Aside from my specific issue described, i know this might be a bold request since extensions can have arbitrary JS code (as far as i am aware). But it seems the default behaviour to download (any) Jupyterhub extension js and integrate it during the VSCode Jupyter Notebook execution. Am i right there ? This might be quite a complex feature to support, since different Jupyterhub instances might supply different extension js code (versions). So loading and using different js code in VSCode per open Notebook and depending on the Jupyterhub kernel used for a specific Notebook would probably need to be supported for best compatibility.

Actual behaviour

When using a Jupyterhub instance with the plotly jupyterlab extension setup i get the following error when accessing the remote kernel while using a username with special characters like user (test): Error downloading from http://localhost:8000/user/user (test)/nbextensions/jupyterlab-plotly/extension.js: Not Found

The adjusted url http://localhost:8000/user/user%20%28test%29/nbextensions/jupyterlab-plotly/extension.js does work for downloading the expected extension code.

Executing a Notebook still works with the remote Kernel, but if one executes a Notebook which makes use of the plotly extension, VSCode downloads the plotly extension js code from a cdn instead of the Jupyterhub server. This might lead to incompatibilities since the version downloaded from the cdn and the version on Jupyterhub differs.

With a "normal" username like normal_user i dont see the same error message and it also does not seem to download the extension code from the cdn but uses the Jupyerhub provided version.

This issue was disovered and might be related to https://github.com/microsoft/vscode-jupyter-hub/issues/53#issuecomment-2030552816.

Steps to reproduce:

  1. Setup or use a local or remote Jupyterhub instance with the jupyterlab extension plotly . The extension can be installed by installing the plotly python package into the base environment of the juypterlab user environment. For my later provided test Notebook code to work, the plotly-resampler python package also is needed. Login in and start the server with a user called user (test). This is quite easy to simulate with the DummyAuthenticator authenticator module which allows any kind of username to be tested.

  2. Setup a API Tokens for authentication on the Jupyterhub instance

  3. Use the url, user name and token and create a new remote Jupyterhub connection

    1. Select existing Jupyterhub-Server
    2. Select enter URL for existing Server
    3. Enter URL for Jupyterhub server. In my local example http://localhost:8000
    4. Enter Username for Jupyterhub server instance. In my local example user (test)
    5. Enter previously generated Jupyterhub server instance token as Password.
    6. Enter a name under which Vscode saves the Jupyterhub server. In my local example JupyterHub (test)
    7. The Jupyterhub connection details are saved but on the first time i try to access it via VSCode i see the error Error downloading from http://localhost:8000/user/user (test)/nbextensions/jupyterlab-plotly/extension.js: Not Found.

  4. Execute the following test Notebook code with the saved remote kernel from JupyterHub (test)

    import plotly.graph_objects as go; import numpy as np
from plotly_resampler import FigureResampler, FigureWidgetResampler

x = np.arange(1_000_000)
noisy_sin = (3 + np.sin(x / 200) + np.random.randn(len(x)) / 10) * x / 1_000

# OPTION 1 - FigureWidgetResampler: dynamic aggregation via `FigureWidget.layout.on_change`
fig = FigureWidgetResampler(go.Figure())
fig.add_trace(go.Scattergl(name='noisy sine', showlegend=True), hf_x=x, hf_y=noisy_sin)

fig
    1. (Can not reproduce this as it only happened once) Get asked by a popup if one wants to download the Jupyterlab extension code since one is using it. Is there a setting i flipped by agreeing the first time ?
    2. Press Agree or something to that effect.

  5. Everything looks as expected now. The plotly graph can be interactively used (zoom in and out etc.). But i see log messages about the extension code being downloaded from a cdn url instead of the Jupyterhub server.

Logs

Output for Jupyter in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Jupyter)

``` Visual Studio Code (1.87.2, undefined, desktop) Jupyter Extension Version: 2024.2.0. Python Extension Version: 2024.2.1. Pylance Extension not installed. Platform: linux (x64). Workspace folder <*****>, Home = <*****> 12:08:31.430 [info] Start refreshing Kernel Picker (1712052511430) 12:08:31.442 [info] Using Pylance 12:08:31.515 [info] Start refreshing Interpreter Kernel Picker 12:08:36.890 [info] Starting Kernel connectToLiveRemoteKernel, 90f4c73e-d252-42af-b12e-4c299d7e00eb for '<*****>Plotly_Test.ipynb' (disableUI=true) 12:08:40.166 [warn] Failed to get activated env vars for /usr/bin/python3 in 515ms 12:08:40.501 [info] Process Execution: /usr/bin/python3 -c "import site;print("USER_BASE_VALUE");print(site.USER_BASE);print("USER_BASE_VALUE");" 12:08:40.586 [info] Process Execution: /usr/bin/python3 -m pip list 12:08:41.018 [info] Started session for kernel connectToLiveRemoteKernel:90f4c73e-d252-42af-b12e-4c299d7e00eb 12:08:41.111 [info] Registering Kernel Completion Provider from kernel Python 3 (testenv) (Plotly_Test.ipynb) for language python 12:08:41.959 [error] Error downloading from http://localhost:8000/user/user (test)/nbextensions/jupyterlab-plotly/extension.js: Not Found 12:08:41.960 [warn] Failed to fetch script sources [TypeError: Cannot read properties of undefined (reading 'replace') at Hc.ourFetch [as fetchImplementation] (~/.vscode/extensions/ms-toolsai.jupyter-hub-2024.4.100/dist/extension.node.js:68102:32) at Hc.downloadFile (~/.vscode/extensions/ms-toolsai.jupyter-2024.2.0-linux-x64/dist/extension.node.js:265:13637) at UT.getWidgetScriptSourceUsingFetch (~/.vscode/extensions/ms-toolsai.jupyter-2024.2.0-linux-x64/dist/extension.node.js:341:86140)] 12:08:41.961 [error] Error downloading from http://localhost:8000/user/user (test)/nbextensions/jupyter-js-widgets/extension.js: Not Found 12:08:43.173 [warn] Disposing old controller startUsingPythonInterpreter:'.jvsc74a57bd0062439a9290cb9c34d8371885646b06c6e09a46aa70f3b088588fb26e63fdf2b./python3./python3.-m#ipykernel_launcher' for view = 'jupyter-notebook' 12:08:43.174 [warn] Disposing old controller startUsingPythonInterpreter:'.jvsc74a57bd0062439a9290cb9c34d8371885646b06c6e09a46aa70f3b088588fb26e63fdf2b./python3./python3.-m#ipykernel_launcher (Interactive)' for view = 'interactive' 12:08:44.282 [info] End refreshing Kernel Picker (1712052511430) 12:08:53.992 [info] Handle Execution of Cells 0 for <*****>Plotly_Test.ipynb 12:08:54.000 [info] Kernel acknowledged execution of cell 0 @ 1712052534000 12:08:54.064 [info] Fetch Script for {"moduleName":"jupyterlab-plotly","moduleVersion":"^5.19.0","requestId":"jupyterlab-plotly:^5.19.0:1712052534060"} 12:08:54.064 [info] Fetch Script for jupyterlab-plotly 12:08:54.065 [info] Searching for Widget Script jupyterlab-plotly#^5.19.0 using cdns jsdelivr.com unpkg.com 12:08:54.149 [info] End cell 0 execution after 0.148s, completed @ 1712052534148, started @ 1712052534000 12:08:54.307 [info] Widget Script jupyterlab-plotly#^5.19.0 found at URI: https://cdn.jsdelivr.net/npm/jupyterlab-plotly@5.19.0/dist/index.js 12:08:54.307 [info] Script for jupyterlab-plotly, is https://cdn.jsdelivr.net/npm/jupyterlab-plotly@5.19.0/dist/index.js from cdn 12:19:10.082 [info] Disposing kernel 90f4c73e-d252-42af-b12e-4c299d7e00eb for notebook <*****>Plotly_Test.ipynb due to selection of another kernel or closing of the notebook 12:19:10.083 [info] Dispose Kernel '<*****>Plotly_Test.ipynb' associated with '<*****>Plotly_Test.ipynb' 12:19:12.089 [info] Start refreshing Kernel Picker (1712053152089) 12:19:12.150 [warn] Disposing old controller startUsingPythonInterpreter:'.jvsc74a57bd0062439a9290cb9c34d8371885646b06c6e09a46aa70f3b088588fb26e63fdf2b./python3./python3.-m#ipykernel_launcher' for view = 'jupyter-notebook' 12:19:12.150 [warn] Disposing old controller startUsingPythonInterpreter:'.jvsc74a57bd0062439a9290cb9c34d8371885646b06c6e09a46aa70f3b088588fb26e63fdf2b./python3./python3.-m#ipykernel_launcher (Interactive)' for view = 'interactive' 12:19:12.151 [info] End refreshing Kernel Picker (1712053152089) ```

quantum-byte commented 4 months ago

Quick addition to the potential impact:

Executing a Notebook still works with the remote Kernel, but if one executes a Notebook which makes use of the plotly extension, VSCode downloads the plotly extension js code from a cdn instead of the Jupyterhub server. This might lead to incompatibilities since the version downloaded from the cdn and the version on Jupyterhub differs.

After looking at the log output again, you seem to identify the plotly version and download the version specific to the plotly python version installed. So code incompatibilities should not be an issue anymore.

But there might still be issues for extensions where you can not fall back to the cdn for download if there is such a thing. And the behaviour is in general different just because of the username used. Username with special chars => CDN download while normal Username => Jupyterhub download.

DonJayamanne commented 4 months ago

Thanks for filing this issue, I will be shipping a new pre-release version of the Jupyter extension this week, you can then install that in VS Code Insiders along with the Jupyter Hub VSIX I will provide & then test it out.

Hopefully this should not be an issue. The version of plotly downloaded from the CDN is based on a version, we do not always download the latest version, The version of the npm package is returned by the python code (module version).

I.e. versions shouldn't be an issue.

quantum-byte commented 4 months ago

Addition 2:

I figured out the setting causing the popup asking for the cdn download not to appear anymore. Its

"jupyter.widgetScriptSources": [
        "jsdelivr.com",
        "unpkg.com"
    ]

The popup looks like this: vscode_jupyter_plotly_download_popup

If it is set even with the normal username the extension code is downloaded from the cdn.

If it is not set it fails to download the extension code from the cdn and actually falls back to the Jupyterhub instance version. Not sure if i missed this the first time i tested it or if there is some other edgecase happening. See this log example.

13:13:30.412 [info] Fetch Script for {"moduleName":"jupyterlab-plotly","moduleVersion":"^5.19.0","requestId":"jupyterlab-plotly:^5.19.0:1712056410411"}
13:13:30.412 [info] Fetch Script for jupyterlab-plotly
13:13:30.413 [warn] Widget Script Source not found for jupyterlab-plotly@^5.19.0 from cdn
13:13:30.656 [info] End cell 0 execution after 0.818s, completed @ 1712056410656, started @ 1712056409838
13:13:30.682 [info] Script for jupyterlab-plotly, is http://localhost:8000/user/normal_user/nbextensions/jupyterlab-plotly/index from remote
quantum-byte commented 4 months ago

Hopefully this should not be an issue. The version of plotly downloaded from the CDN is based on a version, we do not always download the latest version, The version of the npm package is returned by the python code (module version).

I.e. versions shouldn't be an issue.

Yea i saw that also after i opened the Ticket. Even if in the end everything works since the extension scripts usually are downloaded from the cdn with the correct version, i think the issue should still be fixed at some point, because it is an additional inconsistency depending on the username. Also maybe this means in some parts of the VS extension code base besides just this case the wrong encoded url is used and leads to worse problems.

DonJayamanne commented 4 months ago

additional inconsistency depending on the username. Also maybe this means in some parts of the VS extension code base besides just this case the wrong encoded url is used and leads to worse problems.

that is being tracked in a separate issue, and will be fixed. Lets not mix these two,

I figured out the setting causing the popup asking for the cdn download not to appear anymore. Its

Given this is fixed, I'm closing this issue. The issue with regards to encoding has been resolved and is tracked in a separate issue

quantum-byte commented 3 months ago

additional inconsistency depending on the username. Also maybe this means in some parts of the VS extension code base besides just this case the wrong encoded url is used and leads to worse problems.

that is being tracked in a separate issue, and will be fixed. Lets not mix these two,

I figured out the setting causing the popup asking for the cdn download not to appear anymore. Its

Given this is fixed, I'm closing this issue. The issue with regards to encoding has been resolved and is tracked in a separate issue

You mean https://github.com/microsoft/vscode-jupyter-hub/issues/53 ? Because this issue regarding juypter extension download issues still exists in the dev version of the vscode jupyter-hub extension you linked me in https://github.com/microsoft/vscode-jupyter-hub/issues/53#issuecomment-2030552803. The here provided information is with this dev version installed.

Unless you fixed this issue later it still exists in the code base.

DonJayamanne commented 3 months ago

Please try in a few days with the latest version of vscode insiders and pre release version of Jupiter extension The changes are on the main branch