Closed manikanta-dornala closed 1 year ago
rchiodo
commented
1 year ago Thanks for the bug.
Do you have any more information? Like a public repository (or code sample) that causes the issue? How long does the process peg the CPU?
Additionally, can you provide the information outlined here:
https://github.com/microsoft/pylance-release/blob/main/TROUBLESHOOTING.md#filing-an-issue
gyandeeps
commented
1 year ago @rchiodo Facing the same issue, here is log from python language server
gyandeeps
commented
1 year ago @rchiodo Just an FYI, with the preview version v2023.1.31 i am not seeing this issue. I see this issue in v2023.1.30.
rchiodo
commented
1 year ago Might be one of the bugs that Eric fixed recently in Pyright.
ivanistheone
commented
1 year ago I am getting the high CPU usage today for the "Code Helper (Plugin)" process during editing a very simple file: https://github.com/minireference/noBSstatsnotebooks/blob/main/notebooks/plot_helpers.py
This is on Mac OS 11.7.1
If I disable the Pylance plugin, the high CPU usage doesn't occur.
Here are the log I was able to extract:
In another log file I saw the following errors that might be relevant:
and 50 or 100 more of this kind of errors, one for each virtualenv in my Projects folder (I use separate venv for each project, so there are a lot of them!!!)
I did recently run a pip install -U -r reqirements.txt (on an unversioned reqs file) so probably lots of code in the venv has changed, but I've never seen CPU this high.
Here is a pip freeze output for reference
I believe this happened to me once before, and I fixed it by deleting the venv and starting from clean.
Open to recommendations for futher testing/debugging I can do to help diagnose this.
ivanistheone
commented
1 year ago Okay now high CPU usage is in all Python projects:
It seems to be reporting EACCES when trying to touch the venv folder (the location of my local virtual env)
rchiodo
commented
1 year ago @ivanistheone that last issue is a dupe of this one: https://github.com/microsoft/pylance-release/issues/3907
If the python environment cannot be started for some reason, pylance will crash. This is only a bug in the prerelease bits. We should be shipping a fix for this bug today in a new prerelease.
Morilli
commented
1 year ago I'm facing the same issue of the pylance language server periodically reaching high cpu usage, making for an unpresent experience both while navigating any python project and when having vscode in idle in the background.
For what it's worth, I tested on the latest version (2023.2.20) and the latest preview version (2023.2.21), and both versions show this problem. The only thing I'm doing is opening an empty folder with vscode, creating an empty empty.py file and immediately the CPU issues show up.
Here's an example log output from the Python Language Server console with logLevel set to trace:
It's noteworthy that there's always a pause between the LibraryFile: Received fs event 'change' for path 'c:\Users\Moritz\.vscode\extensions\ms-python.vscode-pylance-2023.2.21\dist\.cache\stubs\xxhash.json' line and the following Background analysis message: invalidateAndForceReanalysis.
For some reason, it seems to be doing the same analysis of whatever it's indexing over and over again, even though nothing changes.
I'd be happy to provide more details to help figure this out because as it stands, I effectively cannot use this extension because the pressure on my CPU is too high.
erictraut
commented
1 year ago @Morilli, is this code in a public repo? It's difficult to diagnose these types of problems without a repro case. If it's in a private code base, we will need your help in isolating the problem and producing a repro for us.
Morilli
commented
1 year ago @erictraut what code do you mean? As I wrote above, I can reproduce this CPU issue by opening an empty folder and creating an empty .py file from scratch. No other code is present, unless you mean my local python installation?
erictraut
commented
1 year ago Ah, I missed that. So this is all due to symbol indexing. Then this is either a problem that occurs when indexing a particular library that you have installed or a mismatch in expectations (i.e. you expect indexing to take less computation than it realistically requires). Based on the performance numbers I see in the log, it doesn't appear to be taking that much time. Just a little over a second.
You could try starting with an empty virtual environment and then add libraries to it until you find one that triggers a performance issue.
You can disable symbol indexing if you find it to be taking too much time.
Morilli
commented
1 year ago Okay with what I have figured out so far, the issue seems to be caused solely by my installation of the xxhash package. It's a local installation, not from PyPI, and seems to have some quirk that causes the python language server to rerun its indexing over and over again (this is most likely what LibraryFile: Received fs event 'change' for path 'c:\Users\Moritz\.vscode\extensions\ms-python.vscode-pylance-2023.1.20\dist\.cache\stubs\xxhash.json' is; something is writing to that file and the indexing gets re-queued or something).
I checked previous Pylance versions and apparently everything is fine up until version 2023.1.10; the issue starts with version 2023.1.20.
I've tried to create a simple standalone python package that, when installed, shows this behavior, but have been unsuccessful so far. I'll report back once I manage to get one working.
ivanistheone
commented
1 year ago The process that is using high CPU here 66973 /Applications/Visual Studio Code.app/Contents/Frameworks/Code Helper (Plugin).app/Contents/MacOS/Code Helper (Plugin) ...
Seems to have two sub-processes: /Applications/Visual Studio Code.app/Contents/Frameworks/Code Helper (Plugin).app/Contents/MacOS/Code Helper (Plugin) --ms-enable-electron-run-as-node /Applications/Visual Studio Code.app/Contents/Resources/app/extensions/json-language-features/server/dist/node/jsonServerMain --node-ipc --clientProcessId=66973
and /Applications/Visual Studio Code.app/Contents/Frameworks/Code Helper (Plugin).app/Contents/MacOS/Code Helper (Plugin) --ms-enable-electron-run-as-node /Users/ivan/.vscode/extensions/ms-python.vscode-pylance-2023.2.21/dist/server.bundle.js --cancellationReceive=file:fa09ccf724aaf3ff98ab0bd1444f190008d53ea998 --node-ipc
From the timestamps in the Python and logs Pylance logs it seems the indexing and checks for venvs run quickly (within 10 seconds or so), so I don't think it is an indexing issue.
Could it be some issue with node?
I'm going to try uninstalling vscode app completely and re-installing to see if that helps any.
ivanistheone
commented
1 year ago Okay after removing all files in .vscode and LIbrary/Applicaiton Support/Code I was able to re-install vscode App with Pylance extension and no more high CPU usage.
I suspect there was some interaction with one of the many other extensions I had. I guess it's always good to start with a clean dev env once in a while ;)
ivanistheone
commented
1 year ago Update: the high CPU usage was caused by Jupyter extension. I am able to reliably reproduce now: if Jupyter extension is enabled the CPU goes into 250% spin. After I disable Jupyter extension everythign is back to normal (Pylance works fine).
I suspect you might hear more about it, because for some reason Jupyter extension was installed automatically for me when I chose Python language support.
Let me know if it would be useful to report this issue somewhere else.
rchiodo
commented
1 year ago It would be great if you could log the perf issue on the jupyter extension here: https://github.com/microsoft/vscode-jupyter/issues
Please make sure the jupyter extension is up to date. There were some problems with them integrating with us if all the versions aren't the latest ones.
Morilli
commented
1 year ago Well, I have a reproduction scenario for windows, however I wasn't able to reproduce this with the python from https://www.python.org/downloads/, but just the the ucrt64 python version installed with msys2 from https://www.msys2.org/. I haven't tested any other python environments or versions.
To reproduce:
pacman -S git mingw-w64-ucrt-x86_64-gcc mingw-w64-ucrt-x86_64-python mingw-w64-ucrt-x86_64-python-pip mingw-w64-ucrt-x86_64-python-setuptoolspip install setuptools_scm build wheelgit clone --recursive https://github.com/ifduyue/python-xxhash xxhash && cd xxhash; python -m build -n -w && cd dist && pip install xxhash-3.2.1.dev1-cp310-cp310-mingw_x86_64_ucrt.whlNow, open any folder with vscode and ensure the python interpreter is set to C:\msys64\ucrt64\bin\python.exe and this issue should present itself.
ivanistheone
commented
1 year ago In the interest of resolving this, I will repost a comment from @stephinity from the jupyter thread on high CPU problem:
I am still having the high CPU problem, and today ran the "Extension Bisect" utility in VS Code. After going through the different combinations of active extensions, the CPU is high when Jupyter & Pylance extensions are both active. Having the Jupyter Extension active with Pylance disabled also works fine and at normal CPU.
@Morilli and others with high CPU issue: Do you have the Jupyter extension enabled by any chance?
Perhaps there is a common cause? (node crashes or goes into infinite loop)
Morilli
commented
1 year ago I never tested with the jupyter extension installed, in fact I can reproduce this with only the python+pylance extensions active.
Kurt-von-Laven
commented
1 year ago I am able to reproduce this issue with only the Python v2023.4.1 and Pylance v2023.3.20 extensions active in VSCode 1.76.1 on Ubuntu 22.10. The CPU utilization remains high upon launching VSCode until returning to baseline after about 80 seconds, precisely when the following error appears: "Unable to watch for file changes in this large workspace folder. Please follow the instructions link to resolve this issue." Repeatedly running the following Bash script inspired by this Stack Overflow answer proves that a single VSCode process abruptly (within approximately a second) consumed all available file watchers right around that 80 second mark, explaining why the CPU utilization returns to baseline.
find /proc/*/fd -lname anon_inode:inotify -printf '%hinfo/%f\n' 2>/dev/null |
xargs grep --count '^inotify' |
sort --numeric-sort --field-separator=: --key=2 --reverse |
head
cat /proc/sys/fs/inotify/max_user_watchesThe issue reproduces in all of our repos (e.g., ScribeMD/template-repo) regardless of whether they contain any *.py files. Running poetry install --sync to install our Python dependencies or, alternatively, rm -rf .venv to remove them has no effect on the bug.
dbrakenhoff
commented
1 year ago I'm experiencing the same issue with Pylance causing high CPU usage every few seconds (and memory consumption increasing rapidly to 3+GB) and causing my laptop fan to start spinning loudly.
I've updated the extensions to the latest versions. I've tried adding the relevant info below. VS Codes extension bisect pointed to Pylance as the cause. I've tried disabling the Jupyter extension, and the problem persisted.
See attached log. PythonLanguageServer.zip
bschnurr
commented
1 year ago @dbrakenhoff. did you manually set any extra paths? one strange thing I noticed is that we are watching the pylance dist folder which could trigger re-analysis.
bschnurr
commented
1 year ago now i see are latest scipy typestubs are missing a bunch of files in the scipy\interpolate folder
for now you could try generating your own.
add # pyright: reportMissingTypeStubs=true to the top of grid.py and then run the code action Create Type Stubs for "scipy.interpolate"
this should help a bit with the performance until we update pylance next week with more stubs
bschnurr
commented
1 year ago @dbrakenhoff are you seeing perf issues too on subsequent simple edits ? ie. adding a single space text edit?
bschnurr
commented
1 year ago In the interest of resolving this, I will repost a comment from @stephinity from the jupyter thread on high CPU problem:
I am still having the high CPU problem, and today ran the "Extension Bisect" utility in VS Code. After going through the different combinations of active extensions, the CPU is high when Jupyter & Pylance extensions are both active. Having the Jupyter Extension active with Pylance disabled also works fine and at normal CPU.
@Morilli and others with high CPU issue: Do you have the Jupyter extension enabled by any chance?
Perhaps there is a common cause? (node crashes or goes into infinite loop)
please create a new issue with instructions on how to setup/download the repo. i'm not familiar with template repos.
dbrakenhoff
commented
1 year ago Hi @bschnurr ,
Thanks for the quick reply! My answers are below:
did you manually set any extra paths?
I'm not aware of manually setting any extra paths, not in any .vscode/settings anyway, but I'm not entirely sure what you mean by that.
@dbrakenhoff are you seeing perf issues too on subsequent simple edits ? ie. adding a single space text edit?
Yes, often after a focus change the editor hangs for a second before allowing me to type anything. Also the autocomplete/ctrl+space becomes slow/unresponsive.
this should help a bit with the performance until we update pylance next week with more stubs
And thanks for the suggestion about the Type Stubs. I'm generating them now and will check if that impacts performance. There's a bunch of packages that are missing them, does it make sense to generate them for each of those packages (if at all possible)?
bschnurr
commented
1 year ago Ya, the more typestubs the better.. though you will miss out on some types and return statements no longer being inferred if they dont have annotation in the typestub. I would try out the latest pre-release as well. Pylance v2023.3.41 as well.
dbrakenhoff
commented
1 year ago As for that /home/user/.vscode/extensions* path, I don't know why it is in that list. The rest are either paths that the conda environment activation added, including paths of packages installed in --editable mode, and one path I added myself with some binaries.
erictraut
commented
1 year ago Keep in mind that the "generate stub" feature emits a "first draft" of the stub files. It's typically necessary to do some manual cleanup to produce correct stubs. This requires some knowledge of the library and its behaviors.
dbrakenhoff
commented
1 year ago @bschnurr, installing the pre-release seems to have helped! It's no longer continually rescanning/indexing according the the Python Language Server log. So looking good, I'll check it out for a bit longer, but it seems resolved for my case.
@erictraut, where do these generated stubs end up? And should I remove them somehow when the next release of Pylance includes better(?) stubs?
bschnurr
commented
1 year ago by default they get generated relative to your workspace root in a typings folder
erictraut
commented
1 year ago The generated stubs are (by default) placed in a directory called typings within your project. Once you invest time in generating and manually improving stubs, you typically want to check them in so other collaborators on your project can benefit from them as well.
And yes, you can manually delete them if and when better stubs become available — either bundled with pylance or installed in your Python environment as a stub library.
Kurt-von-Laven
commented
1 year ago please create a new issue with instructions on how to setup/download the repo. i'm not familiar with template repos.
@bschnurr, did you mean this in reply to me? The fact that the repo is a template repo is immaterial. It can simply be cloned, or any of our other repos can be used. Happy to create a new issue if you were, in fact, talking to me.
bschnurr
commented
1 year ago @Kurt-von-Laven yes Kurt.. please do, and I'll take a look
Kurt-von-Laven
commented
1 year ago Thank you! Filed #4177.
judej
commented
1 year ago dupe
Type: Bug
I hear my fan spin on high when this extension is enabled. Using Extension bisect, when its disabled I find that my CPU usage goes down from 25% (of which Code Helper is 100%) to 2%!
Extension version: 2022.12.20 VS Code version: Code 1.74.1 (Universal) (1ad8d514439d5077d2b0b7ee64d2ce82a9308e5a, 2022-12-14T10:33:40.793Z) OS version: Darwin x64 22.2.0 Modes: Sandboxed: No
System Info
|Item|Value| |---|---| |CPUs|Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz (16 x 2400)| |GPU Status|2d_canvas: enabledcanvas_oop_rasterization: disabled_off
direct_rendering_display_compositor: disabled_off_ok
gpu_compositing: enabled
metal: disabled_off
multiple_raster_threads: enabled_on
opengl: enabled_on
rasterization: enabled
raw_draw: disabled_off_ok
skia_renderer: enabled_on
video_decode: enabled
video_encode: enabled
vulkan: disabled_off
webgl: enabled
webgl2: enabled
webgpu: disabled_off| |Load (avg)|3, 4, 4| |Memory (System)|32.00GB (0.48GB free)| |Process Argv|--crash-reporter-id 8c6aa5ea-83b8-48cf-b41e-ad4e64856c36| |Screen Reader|no| |VM|0%|
A/B Experiments
``` vsliv368:30146709 vsreu685:30147344 python383cf:30185419 vspor879:30202332 vspor708:30202333 vspor363:30204092 vslsvsres303:30308271 pythonvspyl392:30443607 vserr242:30382549 pythontb:30283811 vsjup518:30340749 pythonptprofiler:30281270 vshan820:30294714 vstes263cf:30335440 vscoreces:30445986 pythondataviewer:30285071 vscod805cf:30301675 binariesv615:30325510 bridge0708:30335490 bridge0723:30353136 cmake_vspar411:30581797 vsaa593cf:30376535 pythonvs932:30410667 cppdebug:30492333 vsclangdf:30486550 c4g48928:30535728 dsvsc012cf:30540253 azure-dev_surveyonecf:30548226 vscccc:30610679 pyindex848cf:30577861 nodejswelcome1cf:30587006 3biah626:30602489 gswce1:30612156 3d0df643:30613357 f6dab269:30613381 fim-prod:30623723 ```