rapidsai / jupyterlab-nvdashboard

A JupyterLab extension for displaying dashboards of GPU usage.
BSD 3-Clause "New" or "Revised" License
598 stars 78 forks source link

nvdashboard turns a blank page #80

Closed qurainia closed 3 years ago

qurainia commented 4 years ago

image

[I 07:51:50.369 LabApp] JupyterLab extension loaded from /home/aquraini/anaconda3/lib/python3.8/site-packages/jupyterlab
[I 07:51:50.369 LabApp] JupyterLab application directory is /home/aquraini/anaconda3/share/jupyter/lab
[I 07:51:50.375 LabApp] Serving notebooks from local directory: /lustre/fsw/sa/aquraini/claratraindevday
[I 07:51:50.375 LabApp] The Jupyter Notebook is running at:
[I 07:51:50.375 LabApp] http://luna-0103:8893/?token=0e772eb3c6b0062802bc46c65784a4fdb93e22c0f39b3197
[I 07:51:50.375 LabApp]  or http://127.0.0.1:8893/?token=0e772eb3c6b0062802bc46c65784a4fdb93e22c0f39b3197
[I 07:51:50.375 LabApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 07:51:50.388 LabApp]

    To access the notebook, open this file in a browser:
        file:///home/aquraini/.local/share/jupyter/runtime/nbserver-124705-open.html
    Or copy and paste one of these URLs:
        http://luna-0103:8893/?token=0e772eb3c6b0062802bc46c65784a4fdb93e22c0f39b3197
     or http://127.0.0.1:8893/?token=0e772eb3c6b0062802bc46c65784a4fdb93e22c0f39b3197
[I 07:58:48.385 LabApp] 302 GET / (10.248.1.195) 0.56ms
[I 07:58:55.964 LabApp] Build is up to date
[W 07:58:59.833 LabApp] 500 GET /nvdashboard/index.json?1600873129573 (10.248.1.195): could not start nvdashboard in time
[E 07:58:59.917 LabApp] {
      "Host": "10.2.54.51:8893",
      "Connection": "keep-alive",
      "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.102 Safari/537.36",
      "X-Xsrftoken": "2|5710e0f4|fadc9c07b4e4f3a52f12c5a28dbbbe27|1600815235",
      "Authorization": "token 0e772eb3c6b0062802bc46c65784a4fdb93e22c0f39b3197",
      "Content-Type": "application/json",
      "Accept": "*/*",
      "Referer": "http://10.2.54.51:8893/lab?",
      "Accept-Encoding": "gzip, deflate",
      "Accept-Language": "en-US,en;q=0.9",
      "Cookie": "_xsrf=2|5710e0f4|fadc9c07b4e4f3a52f12c5a28dbbbe27|1600815235; username-10-2-54-51-8893=\"2|1:0|10:1600873129|24:username-10-2-54-51-8893|44:MTFkZDM2ZWYwMTMzNGYxYmE5NTA0Njg5Mjk5YTIxZWQ=|0d3037725c1fee342bc01ff156b89a617811a2118a57957f7594c25a37904905\""
    }
[E 07:58:59.917 LabApp] 500 GET /nvdashboard/index.json?1600873129573 (10.248.1.195) 10055.04ms referer=http://10.2.54.51:8893/lab?

other details:

jupyter --version
jupyter core     : 4.6.3
jupyter-notebook : 6.0.3
qtconsole        : 4.7.5
ipython          : 7.16.1
ipykernel        : 5.3.2
jupyter client   : 6.1.6
jupyter lab      : 2.1.5
nbconvert        : 5.6.1
ipywidgets       : 7.5.1
nbformat         : 5.0.7
traitlets        : 4.3.3
jupyter labextension list
JupyterLab v2.1.5
Known labextensions:
   app dir: /home/aquraini/anaconda3/share/jupyter/lab
        jupyterlab-nvdashboard v0.4.0  enabled  OK
 nvidia-smi
Wed Sep 23 08:45:39 2020
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 450.80.00    Driver Version: 450.80.00    CUDA Version: 11.0     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  A100-SXM4-40GB      On   | 00000000:07:00.0 Off |                    0 |
| N/A   26C    P0    52W / 400W |      3MiB / 40537MiB |      0%      Default |
|                               |                      |             Disabled |
+-------------------------------+----------------------+----------------------+
|   1  A100-SXM4-40GB      On   | 00000000:0F:00.0 Off |                    0 |
| N/A   26C    P0    53W / 400W |      3MiB / 40537MiB |      0%      Default |
|                               |                      |             Disabled |
+-------------------------------+----------------------+----------------------+
|   2  A100-SXM4-40GB      On   | 00000000:47:00.0 Off |                    0 |
| N/A   25C    P0    51W / 400W |      3MiB / 40537MiB |      0%      Default |
|                               |                      |             Disabled |
+-------------------------------+----------------------+----------------------+
|   3  A100-SXM4-40GB      On   | 00000000:4E:00.0 Off |                    0 |
| N/A   25C    P0    54W / 400W |      3MiB / 40537MiB |      0%      Default |
|                               |                      |             Disabled |
+-------------------------------+----------------------+----------------------+
|   4  A100-SXM4-40GB      On   | 00000000:87:00.0 Off |                    0 |
| N/A   26C    P0    53W / 400W |      3MiB / 40537MiB |      0%      Default |
|                               |                      |             Disabled |
+-------------------------------+----------------------+----------------------+
|   5  A100-SXM4-40GB      On   | 00000000:90:00.0 Off |                    0 |
| N/A   27C    P0    51W / 400W |      3MiB / 40537MiB |      0%      Default |
|                               |                      |             Disabled |
+-------------------------------+----------------------+----------------------+
|   6  A100-SXM4-40GB      On   | 00000000:B7:00.0 Off |                    0 |
| N/A   27C    P0    52W / 400W |      3MiB / 40537MiB |      0%      Default |
|                               |                      |             Disabled |
+-------------------------------+----------------------+----------------------+
|   7  A100-SXM4-40GB      On   | 00000000:BD:00.0 Off |                    0 |
| N/A   26C    P0    52W / 400W |      3MiB / 40537MiB |      0%      Default |
|                               |                      |             Disabled |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+
storopoli commented 4 years ago

Same problem

Jeffwan commented 4 years ago

same here. I have not tried old versions

Jeffwan commented 4 years ago

I tried 0.3.0 as well, it doesn't work. However, what I get is 404 errors.

[W 14:39:43.319 LabApp] 404 GET /nvdashboard/index.json?1603057183294 (127.0.0.1) 24.36ms referer=http://localhost:8888/lab/workspaces/auto-g?clone
rhangelxs commented 3 years ago

Working solution for me was complete cleaning of browser cache. Including storage tab in safari

JINU8 commented 3 years ago

not working inside docker container

500 GET /nvdashboard/index.json?1603787503989 (10.42.0.1) 5036.52ms referer=http://10.42.0.20:8880/lab?

DenPodolsky commented 3 years ago

Same problem

Jeffwan commented 3 years ago

I will put some time to help fix this issue. Seems this repo is not that active

AndreyRub commented 3 years ago

Same problem here

andiariffin commented 3 years ago

Same problem here. The GPU Dashboard shows nothing.

image

It seems that the problem lies on the bokeh failed to establish the websocket connection.

image

jacobtomlinson commented 3 years ago

Sorry for the delay in responding here folks.

I'm afraid this looks like a duplicate of #24. Please let's continue the discussion over there.

Jeffwan commented 3 years ago

I am actually have a patch to make it fully compatible with jupyterlab 2.0 now. I will file a PR next week