rapidsai / jupyterlab-nvdashboard

A JupyterLab extension for displaying dashboards of GPU usage.
BSD 3-Clause "New" or "Revised" License
588 stars 77 forks source link

wrong uri prefix caused blank content #194

Closed northfun closed 3 months ago

northfun commented 4 months ago

backgroud

We run jupyter images on k8s, all requests from the jupyter frontend are transferred to the specific jupyter pod by a gateway. Normal jupyter widgets send requests which have uri prefix like "user//", so requests can be transferred to the right pod. The jupyter image worked well on local environment.

Here is one pod info:

bml@jupyter-2cfdd9a2a44531c8-0:~$ jupyter labextension list JupyterLab v4.1.8 /opt/conda/share/jupyter/labextensions jupyterlab_pygments v0.3.0 enabled OK (python, jupyterlab_pygments) nbdime-jupyterlab v2.2.0 enabled X jupyterlab-nvdashboard v0.10.00 enabled OK (python, jupyterlab_nvdashboard) jupyterlab-execute-time v3.1.2 enabled OK (python, jupyterlab_execute_time) @jupyterlab/git v0.33.0 enabled X (python, jupyterlab-git) @jupyter-lsp/jupyterlab-lsp v5.1.0 enabled OK (python, jupyterlab-lsp) @jupyter-server/resource-usage v1.0.2 enabled OK (python, jupyter-resource-usage) @jupyter-widgets/jupyterlab-manager v5.0.10 enabled OK (python, jupyterlab_widgets)

The following extensions are outdated: nbdime-jupyterlab @jupyterlab/git

Consider checking if an update is available for these packages.

Other labextensions (built into JupyterLab) app dir: /opt/conda/share/jupyter/lab

Disabled extensions: @jupyterlab/completer-extension:base-service @jupyterlab/fileeditor-extension:language-server @jupyterlab/lsp-extension:settings @jupyterlab/notebook-extension:language-server bml@jupyter-2cfdd9a2a44531c8-0:~$ nvidia-smi Tue May 7 14:55:52 2024
+-----------------------------------------------------------------------------+ | NVIDIA-SMI 470.57.02 Driver Version: 470.57.02 CUDA Version: 11.4 | |-------------------------------+----------------------+----------------------+ | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |===============================+======================+======================| | 0 Tesla T4 Off | 00000000:AF:00.0 Off | 0 | | N/A 41C P0 27W / 70W | 0MiB / 1500MiB | 0% Default | | | | N/A | +-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=============================================================================| | No running processes found |

what I see

Screen Shot 2024-05-07 at 3 00 35 PM

what I find

normal websocket request is as follow

692becb7eeb96034b4cdfdc62286f6fb

nvdashboard is as follow

e49c3c8e33da954431b4abccd2c4f18c

quastion

Is there something I have missed or nvdashboard doesn't support using in cluster mod?

BTW, we use jupyter labhub to start jupyter.

northfun commented 4 months ago

We find the code was fixed in branch-0.11, but the latest version in pypip is 0.10.0. So when will you release 0.11.0, or how can we install nvdashboard from original code?

AjayThorve commented 4 months ago

@northfun It's actually mentioned in the Readme. You can install the latest nightly version through the rapidsai channel:

# nightly version (for jupyterlab>=4)
pip install --extra-index-url https://pypi.anaconda.org/rapidsai-wheels-nightly/simple --pre jupyterlab_nvdashboard