search

microsoft / vscode

Visual Studio Code
https://code.visualstudio.com
MIT License
163.89k stars 29.17k forks source link

Jupyter Notebook Very Slow and Problems with Cell Executions #206119

Open eduardohscosta opened 8 months ago

eduardohscosta commented 8 months ago

Type: Bug

After updating the VSCode version to 1.86.2, I started having problems with Jupyter Notebooks. When running .ipynb the first time, it works as expected, but it starts to crash and slow down after the first run turning impossible to working with it

VS Code version: Code 1.85.2 (8b3775030ed1a69b13e4f4c628c612102e30a681, 2024-01-18T06:40:32.531Z) OS version: Darwin x64 23.1.0 Modes:

System Info |Item|Value| |---|---| |CPUs|Intel(R) Core(TM) i5-1038NG7 CPU @ 2.00GHz (8 x 2000)| |GPU Status|2d_canvas: enabled
canvas_oop_rasterization: disabled_off
direct_rendering_display_compositor: disabled_off_ok
gpu_compositing: enabled
multiple_raster_threads: enabled_on
opengl: enabled_on
rasterization: enabled
raw_draw: disabled_off_ok
video_decode: enabled
video_encode: enabled
vulkan: disabled_off
webgl: enabled
webgl2: enabled
webgpu: enabled| |Load (avg)|7, 7, 10| |Memory (System)|16.00GB (0.72GB free)| |Process Argv|--crash-reporter-id 8a73fa81-c267-4a23-b160-bafaf2ddcd17| |Screen Reader|no| |VM|0%|
Extensions (8) Extension|Author (truncated)|Version ---|---|--- copilot|Git|1.165.0 copilot-chat|Git|0.12.2023120701 python|ms-|2024.1.10531008 vscode-pylance|ms-|2024.2.2 jupyter|ms-|2023.11.1100101639 jupyter-keymap|ms-|1.1.2 jupyter-renderers|ms-|1.0.17 vscode-jupyter-cell-tags|ms-|0.1.8
A/B Experiments ``` vsliv368cf:30146710 vspor879:30202332 vspor708:30202333 vspor363:30204092 vswsl492:30256859 vscorecescf:30445987 vscod805cf:30301675 binariesv615:30325510 vsaa593:30376534 py29gd2263:30899288 c4g48928:30535728 azure-dev_surveyone:30548225 962ge761:30959799 pythongtdpath:30769146 welcomedialog:30910333 pythonidxpt:30866567 pythonnoceb:30805159 asynctok:30898717 pythontestfixt:30902429 pythonregdiag2:30936856 pyreplss1:30897532 pythonmypyd1:30879173 pythoncet0:30885854 pythontbext0:30879054 accentitlementsc:30887149 dsvsc016:30899300 dsvsc017:30899301 dsvsc018:30899302 pydisww1:30959800 d34g3935:30971562 7j2b6412:30969109 ```
paul-rogers commented 8 months ago

Experiencing the same problem. Linux Mint OS. Start VS Code. Begin working on a Python notebook. After some time, check the System Monitor. There are three code processes, each consuming about 35% of CPU (on an 8-CPU machine). Each process consumes about 2 GB of memory. Only six files are open: a MD file, a Python file and four notebook files, each of moderate size.

The problem seems to be related to the help pop-up when typing. For example, my notebook uses many NumPy and MatPlotLib functions. Type "np." and the method choices pop up. After this occurs some number of times (half a dozen, a dozen?), VS Code enters into the high-CPU state. Code execution becomes very slow. In this state, Python debugging in the notebook is either non-operable, or so slow as to seem non-operable.

The workaround is to restart VS Code. When doing so, it takes a long time for VS Code to exit. In fact, so long that a dialog appears explaining that it is taking a long time.

VS Code info:

Version: 1.86.2
Commit: 903b1e9d8990623e3d7da1df3d33db3e42d80eda
Date: 2024-02-13T19:41:37.860Z
Electron: 27.2.3
ElectronBuildId: 26908389
Chromium: 118.0.5993.159
Node.js: 18.17.1
V8: 11.8.172.18-electron.0
OS: Linux x64 5.15.0-94-generic

I'll add more information the next time this occurs. What information would be useful (besides better reproduction steps)?

paul-rogers commented 8 months ago

The problem occurred again today. Here's what I was able to observe. There were two processes running at 100% of a CPU each (13% of total CPU), and another process that, temporarily was running at 64% of total CPU (or about 4 CPUs flat out.) One of the processes used 4.5 GiB of memory, another 2.5 GiB.

The layman's assessment is that something has a major memory leak and is getting itself into an infinite loop.

Details:

PID: 1515800 Memory: 1.9 GiB CPU: 64% (of total), ~300% (of a single CPU) CPU Time: 1:28:35 Command Line: /usr/share/code/code --type=renderer --crashpad-handler-pid=1515763 --enable-crash-reporter=b8373450-4de6-4c25-9955-af1fa298bdef,no_channel --user-data-dir=/home/paul/.config/Code --standard-schemes=vscode-webview,vscode-file --enable-sandbox --secure-schemes=vscode-webview,vscode-file --cors-schemes=vscode-webview,vscode-file --fetch-schemes=vscode-webview,vscode-file --service-worker-schemes=vscode-webview --code-cache-schemes=vscode-webview,vscode-file --app-path=/usr/share/code/resources/app --enable-sandbox --enable-blink-features=HighlightAPI --first-renderer-process --lang=en-US --num-raster-threads=4 --enable-main-frame-before-activation --renderer-client-id=4 --time-ticks-at-unix-epoch=-1707953006574053 --launch-time-ticks=856957257258 --shared-files=v8_context_snapshot_data:100 --field-trial-handle=0,i,7027395513091583484,4112220588306861456,262144 --disable-features=CalculateNativeWinOcclusion,SpareRendererForSitePerProcess --vscode-window-config=vscode:6bf25551-589d-4741-a767-dbe300ab847a

PID: 1515862 Memory: 2.5 Gib CPU: 13% (of total), 100% (of a single CPU) CPU Time: 42:48 Command Line: /usr/share/code/code --type=utility --utility-sub-type=node.mojom.NodeService --lang=en-US --service-sandbox-type=none --dns-result-order=ipv4first --inspect-port=0 --crashpad-handler-pid=1515763 --enable-crash-reporter=b8373450-4de6-4c25-9955-af1fa298bdef,no_channel --user-data-dir=/home/paul/.config/Code --standard-schemes=vscode-webview,vscode-file --enable-sandbox --secure-schemes=vscode-webview,vscode-file --cors-schemes=vscode-webview,vscode-file --fetch-schemes=vscode-webview,vscode-file --service-worker-schemes=vscode-webview --code-cache-schemes=vscode-webview,vscode-file --shared-files=v8_context_snapshot_data:100 --field-trial-handle=0,i,7027395513091583484,4112220588306861456,262144 --disable-features=CalculateNativeWinOcclusion,SpareRendererForSitePerProcess

PID: 1516084 Memory: 4.7 Gib CPU: 13% (of total), 100% (of a single CPU) CPU Time: 39:10 Command Line: /usr/share/code/code /home/paul/.vscode/extensions/ms-python.vscode-pylance-2024.2.2/dist/server.bundle.js --cancellationReceive=file:41882277ebfa02050635c8618ba8ae57604a91738c --node-ipc --clientProcessId=1515862

While copying this information, CPU for the 800 and 084 processes dropped back to 0. However 862 continued to use 12-13% of total CPU (100% of one CPU) consistently. Memory usage for 084 continues to grow, and is now at 5.1 Gib.

At this point, running even a single simple Python function in a notebook takes forever. The Interrupt action won't actually interrupt. The only workaround is to restart.

The "Saving editors with unsaved changes is taking a bit longer" window appears and stays open for a long time (more than 30 seconds).

Hit Cancel, then try to Exit again. "Backing up editors with unsaved changes is taking a bit longer" appears.

Hit cancel, another dialog appears warning about exiting with unsaved changes. Agree to exit anyway.

"Quitting the application is taking a bit longer, Saving working copies" appears and stays open for an extended time. Click "Quit anyway". Code processes exit and CPU load goes back to normal.

For reference, after restart, the same set of windows are open (again, only about 10 files total, all of moderate size). Maximum code process memory usage is <= 500 MB. CPU usage is 0%.

paul-rogers commented 7 months ago

This problem has not occurred recently after two VS Code updates. It is impossible to prove a negative, so perhaps I've just not hit the particular scenario. But, at least, I've not had VS Code peg the CPU recently.

Current VS Code About Output:

Version: 1.87.1
Commit: 1e790d77f81672c49be070e04474901747115651
Date: 2024-03-06T00:21:07.287Z
Electron: 27.3.2
ElectronBuildId: 26836302
Chromium: 118.0.5993.159
Node.js: 18.17.1
V8: 11.8.172.18-electron.0
OS: Linux x64 5.15.0-94-generic
ale-dg commented 7 months ago

Hi,

I just ran a notebook and monitored the usage of the memory in my Mac and these were the highest. I can understand python using a large memory, although I can't understand why the Code Helper use so much of it. During the execution of the notebook all the "code helpers" jump between 300MB to sometimes 1GB. When I close VSCode, all that memory is released, so basically it saves up to 1.5 GB of memory (if not even more).

Best

Screenshot 2024-03-09 at 6 29 10 PM

rebornix commented 7 months ago

Reading the CPU info

Command Line: /usr/share/code/code /home/paul/.vscode/extensions/ms-python.vscode-pylance-2024.2.2/dist/server.bundle.js --cancellationReceive=file:41882277ebfa02050635c8618ba8ae57604a91738c --node-ipc --clientProcessId=1515862

and the screenshot

image

They seem to be related to https://github.com/microsoft/pylance-release/issues/5614. To ensure we understand if it's caused by Pylance, can anyone try disable the Pylance extension and see if you can reproduce?

ale-dg commented 7 months ago

Hi @rebornix,

Please see https://github.com/microsoft/vscode-jupyter/issues/15351#issuecomment-1996127354 and https://github.com/microsoft/vscode-jupyter/issues/14459#issuecomment-1996093504

Even though there might be some issues with Pylance (I have shared some logs there as well), I have shared some comments and logs with @DonJayamanne about Jupyter.

I am not sure how should we follow-up on this, because there are a lot of issues open which apparently point in the same direction.

Best

DonJayamanne commented 7 months ago

@rebornix @ale-dg /cc

If you run into issues without the Pylance extension & without MD cells please share logs here Note, please disable all extensions when attempting to repro the issue, helps us isolate the problems.

Also, please share verbose logs (as follows):