search

microsoft / vscode-jupyter

VS Code Jupyter extension
https://marketplace.visualstudio.com/items?itemName=ms-toolsai.jupyter
MIT License
1.28k stars 287 forks source link

ipywidgets cause cell to run 30x+ times slower #15699

Closed movy closed 1 month ago

movy commented 4 months ago

Applies To

What happened?

Since several versions ago, cells that involve ipywidgets code, run many times slower than cells with similar code w/o ipywidgets. When running the same notebook in browser jupyter lab, cells with and w/o ipywidgets execute immediately.

I tried disabling extensions, run CPU profiling etc -- there is no clear indication of any direct culprit, which makes me think it's either Jupyter extension or VSCode itself.

The attached notebook involves simple code, so difference between execution times of 0.065s and 2.623s does not seem significant, but in real-world scenarios ipywidgets cell execution takes MINUTES, often with subsequent hanging of VSCode remote server, i.e. causing sporadic reconnects despite stable connection, and eventually asking to reload window.

CPU profiling logs and notebook are attached below.

CPU-20240518T073123.245Z.cpuprofile.txt.zip

ipwidgets_test.ipynb.zip

VS Code Version

Version: 1.89.1 (Universal) Commit: dc96b837cf6bb4af9cd736aa3af08cf8279f7685 Date: 2024-05-07T05:14:24.611Z Electron: 28.2.8 ElectronBuildId: 27744544 Chromium: 120.0.6099.291 Node.js: 18.18.2 V8: 12.0.267.19-electron.0 OS: Darwin x64 22.6.0

Jupyter Extension Version

v2024.4.0 and v2024.4.2024042601 (pre-release)

Jupyter logs

Visual Studio Code (1.89.1, ssh-remote, desktop)
Jupyter Extension Version: 2024.4.0.
Python Extension Version: 2024.6.0.
Pylance Extension Version: 2024.5.1.
Platform: linux (x64).
Workspace folder ~/jesse-bot, Home = /home/ubuntu
15:48:46.709 [info] Telemetry level is off
15:48:46.709 [info] Experiments are disabled, only manually opted experiments are active.
15:48:47.762 [info] Starting Kernel startUsingPythonInterpreter, .jvsc74a57bd0ead1b95f633dc9c51826328e1846203f51a198c6fb5f2884a80417ba131d4e82./bin/python3.11./bin/python3.11.-m#ipykernel_launcher  (Python Path: /bin/python3.11, Unknown, Python 3.11.6, 3.11.6) for '/Volumes/Backup/home/Downloads/peaks_detection_test.ipynb' (disableUI=true)
15:48:47.763 [info] Starting Kernel startUsingPythonInterpreter, .jvsc74a57bd0ead1b95f633dc9c51826328e1846203f51a198c6fb5f2884a80417ba131d4e82./bin/python3.11./bin/python3.11.-m#ipykernel_launcher  (Python Path: /bin/python3.11, Unknown, Python 3.11.6, 3.11.6) for '~/jesse-bot/Research-Backtest-MA.ipynb' (disableUI=true)
15:48:47.785 [error] Failed to validate Password info D [FetchError]: request to http://localhost:8888/ failed, reason: connect ECONNREFUSED 127.0.0.1:8888
    at ClientRequest.<anonymous> (/home/~/.vscode-server/extensions/ms-toolsai.jupyter-2024.4.0-linux-x64/dist/node_modules/node-fetch.js:1:306417)
    at ClientRequest.emit (node:events:517:28)
    at Socket.socketErrorListener (node:_http_client:501:9)
    at Socket.emit (node:events:517:28)
    at emitErrorNT (node:internal/streams/destroy:151:8)
    at emitErrorCloseNT (node:internal/streams/destroy:116:3)
    at processTicksAndRejections (node:internal/process/task_queues:82:21) {
  type: 'system',
  errno: 'ECONNREFUSED',
  code: 'ECONNREFUSED'
}
15:48:47.789 [error] Failed to validate Password info D [FetchError]: request to http://localhost:8888/ failed, reason: connect ECONNREFUSED 127.0.0.1:8888
    at ClientRequest.<anonymous> (/home/~/.vscode-server/extensions/ms-toolsai.jupyter-2024.4.0-linux-x64/dist/node_modules/node-fetch.js:1:306417)
    at ClientRequest.emit (node:events:517:28)
    at Socket.socketErrorListener (node:_http_client:501:9)
    at Socket.emit (node:events:517:28)
    at emitErrorNT (node:internal/streams/destroy:151:8)
    at emitErrorCloseNT (node:internal/streams/destroy:116:3)
    at processTicksAndRejections (node:internal/process/task_queues:82:21) {
  type: 'system',
  errno: 'ECONNREFUSED',
  code: 'ECONNREFUSED'
}
15:48:47.847 [error] Error fetching kernels from http://localhost:8888/ (localhost): FetchError: request to http://localhost:8888/api/kernels?1716022127788 failed, reason: connect ECONNREFUSED 127.0.0.1:8888
    at ClientRequest.<anonymous> (/home/~/.vscode-server/extensions/ms-toolsai.jupyter-2024.4.0-linux-x64/dist/node_modules/node-fetch.js:1:306417)
    at ClientRequest.emit (node:events:517:28)
    at Socket.socketErrorListener (node:_http_client:501:9)
    at Socket.emit (node:events:517:28)
    at emitErrorNT (node:internal/streams/destroy:151:8)
    at emitErrorCloseNT (node:internal/streams/destroy:116:3)
    at processTicksAndRejections (node:internal/process/task_queues:82:21)
15:48:47.848 [error] Error fetching kernels from http://localhost:8888/ (localhost): FetchError: request to http://localhost:8888/api/kernels?1716022127791 failed, reason: connect ECONNREFUSED 127.0.0.1:8888
    at ClientRequest.<anonymous> (/home/~/.vscode-server/extensions/ms-toolsai.jupyter-2024.4.0-linux-x64/dist/node_modules/node-fetch.js:1:306417)
    at ClientRequest.emit (node:events:517:28)
    at Socket.socketErrorListener (node:_http_client:501:9)
    at Socket.emit (node:events:517:28)
    at emitErrorNT (node:internal/streams/destroy:151:8)
    at emitErrorCloseNT (node:internal/streams/destroy:116:3)
    at processTicksAndRejections (node:internal/process/task_queues:82:21)
15:48:47.848 [error] Jupyter Lab Helper:getKernelSpecs failure:  Error: Poll (@jupyterlab/services:KernelSpecManager#specs) is disposed.
    at v.dispose (/home/~/.vscode-server/extensions/ms-toolsai.jupyter-2024.4.0-linux-x64/dist/node_modules/@jupyterlab/services.js:11:10935)
    at ni.dispose (/home/~/.vscode-server/extensions/ms-toolsai.jupyter-2024.4.0-linux-x64/dist/node_modules/@jupyterlab/services.js:11:29299)
    at r.dispose (/home/~/.vscode-server/extensions/ms-toolsai.jupyter-2024.4.0-linux-x64/dist/extension.node.js:191:14050)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async Promise.all (index 0)
    at __.listKernelsFromConnection (/home/~/.vscode-server/extensions/ms-toolsai.jupyter-2024.4.0-linux-x64/dist/extension.node.js:296:20211)
    at /home/~/.vscode-server/extensions/ms-toolsai.jupyter-2024.4.0-linux-x64/dist/extension.node.js:296:17636
15:48:47.850 [error] Jupyter Lab Helper:getKernelSpecs failure:  Error: Poll (@jupyterlab/services:KernelSpecManager#specs) is disposed.
    at v.dispose (/home/~/.vscode-server/extensions/ms-toolsai.jupyter-2024.4.0-linux-x64/dist/node_modules/@jupyterlab/services.js:11:10935)
    at ni.dispose (/home/~/.vscode-server/extensions/ms-toolsai.jupyter-2024.4.0-linux-x64/dist/node_modules/@jupyterlab/services.js:11:29299)
    at r.dispose (/home/~/.vscode-server/extensions/ms-toolsai.jupyter-2024.4.0-linux-x64/dist/extension.node.js:191:14050)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async Promise.all (index 0)
    at __.listKernelsFromConnection (/home/~/.vscode-server/extensions/ms-toolsai.jupyter-2024.4.0-linux-x64/dist/extension.node.js:296:20211)
    at /home/~/.vscode-server/extensions/ms-toolsai.jupyter-2024.4.0-linux-x64/dist/extension.node.js:296:17636
15:48:47.856 [error] Failed to validate Password info D [FetchError]: request to http://localhost:8888/ failed, reason: connect ECONNREFUSED 127.0.0.1:8888
    at ClientRequest.<anonymous> (/home/~/.vscode-server/extensions/ms-toolsai.jupyter-2024.4.0-linux-x64/dist/node_modules/node-fetch.js:1:306417)
    at ClientRequest.emit (node:events:517:28)
    at Socket.socketErrorListener (node:_http_client:501:9)
    at Socket.emit (node:events:517:28)
    at emitErrorNT (node:internal/streams/destroy:151:8)
    at emitErrorCloseNT (node:internal/streams/destroy:116:3)
    at processTicksAndRejections (node:internal/process/task_queues:82:21) {
  type: 'system',
  errno: 'ECONNREFUSED',
  code: 'ECONNREFUSED'
}
15:48:47.858 [error] Failed to validate Password info D [FetchError]: request to http://localhost:8888/ failed, reason: connect ECONNREFUSED 127.0.0.1:8888
    at ClientRequest.<anonymous> (/home/~/.vscode-server/extensions/ms-toolsai.jupyter-2024.4.0-linux-x64/dist/node_modules/node-fetch.js:1:306417)
    at ClientRequest.emit (node:events:517:28)
    at Socket.socketErrorListener (node:_http_client:501:9)
    at Socket.emit (node:events:517:28)
    at emitErrorNT (node:internal/streams/destroy:151:8)
    at emitErrorCloseNT (node:internal/streams/destroy:116:3)
    at processTicksAndRejections (node:internal/process/task_queues:82:21) {
  type: 'system',
  errno: 'ECONNREFUSED',
  code: 'ECONNREFUSED'
}
15:48:47.865 [error] Error fetching kernels from http://localhost:8888/ (localhost): FetchError: request to http://localhost:8888/api/kernels?1716022127857 failed, reason: connect ECONNREFUSED 127.0.0.1:8888
    at ClientRequest.<anonymous> (/home/~/.vscode-server/extensions/ms-toolsai.jupyter-2024.4.0-linux-x64/dist/node_modules/node-fetch.js:1:306417)
    at ClientRequest.emit (node:events:517:28)
    at Socket.socketErrorListener (node:_http_client:501:9)
    at Socket.emit (node:events:517:28)
    at emitErrorNT (node:internal/streams/destroy:151:8)
    at emitErrorCloseNT (node:internal/streams/destroy:116:3)
    at processTicksAndRejections (node:internal/process/task_queues:82:21)
15:48:47.865 [error] Error fetching kernels from http://localhost:8888/ (localhost): FetchError: request to http://localhost:8888/api/kernels?1716022127859 failed, reason: connect ECONNREFUSED 127.0.0.1:8888
    at ClientRequest.<anonymous> (/home/~/.vscode-server/extensions/ms-toolsai.jupyter-2024.4.0-linux-x64/dist/node_modules/node-fetch.js:1:306417)
    at ClientRequest.emit (node:events:517:28)
    at Socket.socketErrorListener (node:_http_client:501:9)
    at Socket.emit (node:events:517:28)
    at emitErrorNT (node:internal/streams/destroy:151:8)
    at emitErrorCloseNT (node:internal/streams/destroy:116:3)
    at processTicksAndRejections (node:internal/process/task_queues:82:21)
15:48:47.866 [error] Jupyter Lab Helper:getKernelSpecs failure:  Error: Poll (@jupyterlab/services:KernelSpecManager#specs) is disposed.
    at v.dispose (/home/~/.vscode-server/extensions/ms-toolsai.jupyter-2024.4.0-linux-x64/dist/node_modules/@jupyterlab/services.js:11:10935)
    at ni.dispose (/home/~/.vscode-server/extensions/ms-toolsai.jupyter-2024.4.0-linux-x64/dist/node_modules/@jupyterlab/services.js:11:29299)
    at r.dispose (/home/~/.vscode-server/extensions/ms-toolsai.jupyter-2024.4.0-linux-x64/dist/extension.node.js:191:14050)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async Promise.all (index 0)
    at __.listKernelsFromConnection (/home/~/.vscode-server/extensions/ms-toolsai.jupyter-2024.4.0-linux-x64/dist/extension.node.js:296:20211)
    at /home/~/.vscode-server/extensions/ms-toolsai.jupyter-2024.4.0-linux-x64/dist/extension.node.js:296:17636
15:48:47.866 [error] UniversalRemoteKernelFinder: Failed to get kernels without cache FetchError: request to http://localhost:8888/api/kernels?1716022127857 failed, reason: connect ECONNREFUSED 127.0.0.1:8888
    at ClientRequest.<anonymous> (/home/~/.vscode-server/extensions/ms-toolsai.jupyter-2024.4.0-linux-x64/dist/node_modules/node-fetch.js:1:306417)
    at ClientRequest.emit (node:events:517:28)
    at Socket.socketErrorListener (node:_http_client:501:9)
    at Socket.emit (node:events:517:28)
    at emitErrorNT (node:internal/streams/destroy:151:8)
    at emitErrorCloseNT (node:internal/streams/destroy:116:3)
    at processTicksAndRejections (node:internal/process/task_queues:82:21)
15:48:47.867 [error] Jupyter Lab Helper:getKernelSpecs failure:  Error: Poll (@jupyterlab/services:KernelSpecManager#specs) is disposed.
    at v.dispose (/home/~/.vscode-server/extensions/ms-toolsai.jupyter-2024.4.0-linux-x64/dist/node_modules/@jupyterlab/services.js:11:10935)
    at ni.dispose (/home/~/.vscode-server/extensions/ms-toolsai.jupyter-2024.4.0-linux-x64/dist/node_modules/@jupyterlab/services.js:11:29299)
    at r.dispose (/home/~/.vscode-server/extensions/ms-toolsai.jupyter-2024.4.0-linux-x64/dist/extension.node.js:191:14050)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async Promise.all (index 0)
    at __.listKernelsFromConnection (/home/~/.vscode-server/extensions/ms-toolsai.jupyter-2024.4.0-linux-x64/dist/extension.node.js:296:20211)
    at /home/~/.vscode-server/extensions/ms-toolsai.jupyter-2024.4.0-linux-x64/dist/extension.node.js:296:17636
15:48:47.867 [error] UniversalRemoteKernelFinder: Failed to get kernels without cache FetchError: request to http://localhost:8888/api/kernels?1716022127859 failed, reason: connect ECONNREFUSED 127.0.0.1:8888
    at ClientRequest.<anonymous> (/home/~/.vscode-server/extensions/ms-toolsai.jupyter-2024.4.0-linux-x64/dist/node_modules/node-fetch.js:1:306417)
    at ClientRequest.emit (node:events:517:28)
    at Socket.socketErrorListener (node:_http_client:501:9)
    at Socket.emit (node:events:517:28)
    at emitErrorNT (node:internal/streams/destroy:151:8)
    at emitErrorCloseNT (node:internal/streams/destroy:116:3)
    at processTicksAndRejections (node:internal/process/task_queues:82:21)
15:48:49.650 [warn] Failed to get activated env vars for /bin/python3.11 in 1888ms
15:48:49.651 [warn] Failed to get activated env vars for /bin/python3.11 in 1878ms
15:48:49.652 [warn] Failed to get activated env vars for /bin/python3.11 in 1300ms
15:48:49.663 [info] Process Execution: /bin/python3.11 -c "import site;print("USER_BASE_VALUE");print(site.USER_BASE);print("USER_BASE_VALUE");"
15:48:49.684 [info] Process Execution: /bin/python3.11 -m pip list
15:48:49.692 [info] Process Execution: /bin/python3.11 -c "import ipykernel; print(ipykernel.__version__); print("5dc3a68c-e34e-4080-9c3e-2a532b2ccb4d"); print(ipykernel.__file__)"
15:48:49.701 [info] Process Execution: /bin/python3.11 -c "import ipykernel; print(ipykernel.__version__); print("5dc3a68c-e34e-4080-9c3e-2a532b2ccb4d"); print(ipykernel.__file__)"
15:48:49.712 [info] Process Execution: /bin/python3.11 -m ipykernel_launcher --f=/home/~/.local/share/jupyter/runtime/kernel-v2-1072876pO8XAKQf7V0H.json
    > cwd: //home/~/jesse-bot
15:48:49.722 [info] Process Execution: /bin/python3.11 -m ipykernel_launcher --f=/home/~/.local/share/jupyter/runtime/kernel-v2-1072876uFIivB24Vdbv.json
    > cwd: ~/jesse-bot
15:48:50.232 [info] Process Execution: /bin/python3.11 /home/~/.vscode-server/extensions/ms-toolsai.jupyter-2024.4.0-linux-x64/pythonFiles/printJupyterDataDir.py
15:49:36.740 [info] Handle Execution of Cells 0,1,2 for /Volumes/Backup/home/Downloads/peaks_detection_test.ipynb
15:49:37.146 [info] Cell 0 completed in 0.394s (start: 1716022176752, end: 1716022177146)
15:49:37.237 [info] Cell 1 completed in 0.065s (start: 1716022177172, end: 1716022177237)
15:49:39.865 [info] Cell 2 completed in 2.623s (start: 1716022177242, end: 1716022179865)

Coding Language and Runtime Version

Python 3.11.6

Running Jupyter locally or remotely?

Remote

DonJayamanne commented 2 months ago

@movy I'm so sorry for missing this issue and never getting back. Are you still experiencing the same issues. I have had a look at the ipynb and the logs & haven't been able to find anything that could cause the delays. Please can you run the command Developer: Measure Extension Host Latency and let me know what you get there.

often with subsequent hanging of VSCode remote server, i.e. causing sporadic reconnects despite stable connection,

This seems to indicate there's another issue, and its manifesting itself in the form of delayed execution.

but in real-world scenarios ipywidgets cell execution takes MINUTES, often with subsequent hanging of

Can you try this after

Please try the same that runs for minutes, as opposed to a few seconds.

vs-code-engineering[bot] commented 1 month ago

This issue has been closed automatically because it needs more information and has not had recent activity. See also our issue reporting guidelines.

Happy Coding!