pylance causing 100% CPU utilization frequently

AbdealiLoKo commented 1 year ago

Note - I know this is a generic issue which is there in the issue tracker. But I find this happens to me quite frequently. And I have checked a bunch of the previous issues were closed because logging information and troubleshoot information was not provided. So, I created a new issue for my case ...

Environment data

Language Server version: v2023.9.20
OS and version: Windows 10 with Ubuntu 22.04.3 on WSL2
Python version (& distribution if applicable, e.g. Anaconda): Python 3.7.16 with miniconda

VSCode version information: Version: 1.82.2 (user setup) Commit: abd2f3db4bdb28f9e95536dfa84d8479f1eb312d Date: 2023-09-14T05:55:25.390Z Electron: 25.8.1 ElectronBuildId: 23779380 Chromium: 114.0.5735.289 Node.js: 18.15.0 V8: 11.4.183.29-electron.0 OS: Windows_NT x64 10.0.19045

Code Snippet

I open vscode using code . in my WSL and when it starts it makes my laptop laggy and slow. Even if I wait for 5-7 mins for it to do all its initialization stuff - the laptop is still slow and laggy (Moving the cursor is laggy).

Before vscode is on: novscode-wsl

After vscode has been on for 5mins (throughout this time it has 100% CPU util): after-vscode-running-for-5mins NOTE: The high CPU processes are all pylance processes when I go to the right in htop

Logs

pylance logs: https://drive.google.com/file/d/1uocJdR2m2YfghvaLK6Ops35uSdX8Pvbj/view?usp=sharing

debonte commented 1 year ago

@AbdealiLoKo is your source code available in a public repository? Or could you provide it to us?

AbdealiLoKo commented 1 year ago

Hi @debonte the code is not public And would be complicated for me to provide it to you

But I can run commands and try things as required OR I can get on a call and screen share ?

Also wanted to add some more observations:

This is not a new thing I am observing. I've been having issues with vscode for over 2 months.
I have been trying to debug this myself in many ways.

The exact issue I have found is: When I have vscode (in wsl) open in the background - my laptop starts heating up + cursor hangs up + laptop fan goes into overdrive.
This does not happen every day. Happens 2-3 times a week.

If I see this early and close vscode - things go back to normal
If I do not notice it and vscode remains open - Eventually my laptops is burning and wsl --shutdown is unable to stop the WSL. Have to restart the machine.

When this happened, sometimes I kept htop open in WSL to observe what was going on.
And noticed that the processes for angular/typescript extension and pylance extension were taking up high CPU. Steps I took about 1 month ago:

I enabled tracing and noticed that flake8 was taking time (we use a lot of flake8 extensions)
So, I moved our setup to black+ruff to avoid any speed issues with flake8

But I can still see pylance spiking up my CPU. The one I reported was one such example which I was trying to code and found that 5 mins after starting vscode syntax highlighting and stuff had not yet showed up. So, I started debugging and got clear logs for the bug report I raised.

JohnathanGander commented 1 year ago

I'm having similar issues. Creating a blank file causes an increase of 30% CPU usage. It looks like it successfully looks up imports but color is never applied for different levels of the program. Opening a Jupyter notebook causes an increase of 75% CPU usage. Prior to this Pylance was working properly for .py files but would periodically stop working for .ipynb files. When a notebook is open I get the notification in VSCode that Pylance is trying to parse my documents, which it never successfully does. Disabling the Pylance extension returns CPU usage to normal levels.

Windows 10

VS Code information: Version: 1.82.2 (user setup) Commit: abd2f3db4bdb28f9e95536dfa84d8479f1eb312d Date: 2023-09-14T05:55:25.390Z Electron: 25.8.1 ElectronBuildId: 23779380 Chromium: 114.0.5735.289 Node.js: 18.15.0 V8: 11.4.183.29-electron.0 OS: Windows_NT x64 10.0.19045

debonte commented 1 year ago

@JohnathanGander, please open a separate issue. With logs and ideally code that repros this. If they end up being the same issue we can close one of them later.

debonte commented 1 year ago

@AbdealiLoKo, the thing that stood out to me in your log was this. Analyzing test_model_engine.py is taking almost two minutes.

2023-09-26 19:30:21.936 [info] [Info  - 7:30:21 PM] (14554) [BG(1)] Long operation: checking: /home/abdealiloko/corridor/corridor-platforms/apps/corridor-api/corridor_api_test/unit/engines/test_model_engine.py (104602ms)

Without access to your source code it will be impossible for us to reproduce and investigate this.

Can you try to bisect that file to determine which piece of code is resulting in the long analysis times and the provide us with a minimal repro?

Tatsh commented 1 year ago

I am getting this issue with my macprefs project. I am on an M1 Max Mac. I disabled every extension except Pylance.

trace.log

From ps:

andrew.udvare.-nd 44224  99.8  0.3 1589261024 223328   ??  R    10:31am   5:39.47
/Applications/MacPorts/Visual Studio Code.app/Contents/Frameworks/Code Helper (Plugin).app/Contents/MacOS/Code Helper (Plugin) --ms-enable-electron-run-as-node
/Users/andrew.udvare.-nd/.vscode/extensions/ms-python.vscode-pylance-2023.9.20/dist/server.bundle.js
--cancellationReceive=file:ce620fa2fd18ded3ebe129902ce7b233cbea68b9db --node-ipc --clientProcessId=44173

Python 3.11 provided by MacPorts

To reproduce:

Clone macprefs
Open the folder in VS Code
Open a Python file (like macprefs/command.py)
View htop or Activity Monitor, sorting by CPU %

Tatsh commented 1 year ago

Downgrading Pylance to 2023.9.10 is a workaround.

JohnYeung-dojjy commented 1 year ago

I encountered similar issue, downgrading Pylance did not help much The problem seems to be type check setting, I had type checking mode 'basic' on, turning it off solved the issue for me.

JohnYeung-dojjy commented 1 year ago

Update: The cause might be me editing a ~6000 lines python file (it is a config file) on Azure cloud, maybe it is just the cpu being unable to support pylance's computation?

github-actions[bot] commented 11 months ago

Hey @debonte, this issue might need further attention.

@AbdealiLoKo, you can help us out by closing this issue if the problem no longer exists, or adding more information.

AbdealiLoKo commented 11 months ago

I am not able to reproduce this anymore - my WSL still hangs once in a week or so. I'm reasonably sure it is with something in vscode - as I don't really have anything else open in the WSL. And 90% of the time I am working on python ... But I cannot confirm if it is pylance cause it happens randomly and I don't have any monitoring that can help with that :(

pdonorio commented 10 months ago

Hey everyone, found my self with the 100% CPU of the "Code Helper (Plugin)" today. Killing it / restarting didn't help, and closing VScode had it still running.

After many time investigating I've learned you can check in the PID process the command and it was pointing to Pylance... found this issue, but no real solutions, until this SO lead me to think of looking into the logs (which after some more digging it's possible in Python: Show Language Server Output in VScode settings).

The logs showed that the server was chunking too many files and timing out.

Only by creating a pyrightconfig.json at the root of my repo and setting specific includes on where to look for, I was able to get the server / code helper back to normal.

See sample config in: https://github.com/microsoft/pyright/blob/main/docs/configuration.md#sample-config-file

Hope this helps others.

b-per commented 9 months ago

I had the same problem. I am on MacOS.

With Pylance v2023.12.1 the logs were

2024-01-11 11:27:51.057 [info] [Error - 11:27:51] (40300) Enumeration of workspace source files is taking longer than 10 seconds.
This may be because:
* You have opened your home directory or entire hard drive as a workspace
* Your workspace contains a very large number of directories and files
* Your workspace contains a symlink to a directory with many files
* Your workspace is remote, and file enumeration is slow
To reduce this time, open a workspace directory with fewer files or add a pyrightconfig.json configuration file with an "exclude" section to exclude subdirectories from your workspace. For more details, refer to https://github.com/microsoft/pyright/blob/main/docs/configuration.md.

I then downgraded to Pylance v2023.11.10, killed the 100 CPU process and my CPU usage is now fine.

Now, in the Pylance logs, I get the following error:

2024-01-11 11:36:45.180 [info] [Warn  - 11:36:45] (41532) Skipping recursive symlink "mypath1" -> "mypath2"

So, in my case, it seems to be that recursive symlinks are skipped in v2023.11.10 but are not skipped in v2023.12.1, generating the issue in the latest version.

debonte commented 9 months ago

I'm going to close this issue since @AbdealiLoKo is no longer able to repro it.

@b-per, since your issue seems to be symlink-related, please upgrade to yesterday's prerelease build (2023.12.104) and see if the issue is resolved. We made a big change to how we handle file URIs in that release.

If anyone else is still experiencing similar issues, please also upgrade to the latest prerelease -- if your issue still repros there, please open a new issue.

b-per commented 9 months ago

Thanks. I installed the pre-release and it is not fixed though:

2024-01-24 13:48:42.981 [info] [Info  - 13:48:42] (70266) Pylance language server 2023.12.104 (pyright version 1.1.347, commit 9ec6beba) starting
2024-01-24 13:48:42.982 [info] [Info  - 13:48:42] (70266) Server root directory: file:///path/.vscode/extensions/ms-python.vscode-pylance-2023.12.104/dist
2024-01-24 13:48:42.985 [info] [Info  - 13:48:42] (70266) Starting service instance "abcd"
2024-01-24 13:48:43.006 [info] [Info  - 13:48:43] (70266) Setting pythonPath for service "abcd": "/path/bin/python"
2024-01-24 13:48:43.007 [info] [Info  - 13:48:43] (70266) Setting environmentName for service "abcd": "3.9.17 (venvname)"
2024-01-24 13:48:43.007 [info] [Info  - 13:48:43] (70266) Loading configuration file at /path/pyrightconfig.json
2024-01-24 13:48:43.051 [info] [Info  - 13:48:43] (70266) Assuming Python version 3.9
2024-01-24 13:48:43.051 [info] [Info  - 13:48:43] (70266) No include entries specified; assuming /path
2024-01-24 13:48:53.102 [info] [Error - 13:48:53] (70266) Enumeration of workspace source files is taking longer than 10 seconds.
This may be because:
* You have opened your home directory or entire hard drive as a workspace
* Your workspace contains a very large number of directories and files
* Your workspace contains a symlink to a directory with many files
* Your workspace is remote, and file enumeration is slow
To reduce this time, open a workspace directory with fewer files or add a pyrightconfig.json configuration file with an "exclude" section to exclude subdirectories from your workspace. For more details, refer to https://github.com/microsoft/pyright/blob/main/docs/configuration.md.

I will open a new issue when I have time but will stick to v2023.11.10 for now

debonte commented 9 months ago

@b-per, when you open the issue, if you can provide access to a repo that reproduces the problem that would be appreciated. Without being able to repro this issue ourselves, it's unlikely that we'll make progress on this.

pdonorio commented 6 months ago

Another update:

we can now set exclude / include in JSON config of VS code directly (ref)
having a monorepo I also had to use higher index limit

(in particular I had to exclude the folders with symlinks, that looked like causing some infinite loops. I can't reproduce in a small repo, but it's happening in the big monorepo)

microsoft / pylance-release