Pylance using a lot of memory

ale-dg commented 8 months ago

Hi,

I was checking the activity monitor in my Mac these days and it always shows "Code Helper (plugin)" with over 1 GB of memory usage. When checking through the Terminal which process is it, it turns out it is Pylance the one which is using this much memory. This sometimes causes a lag in VSCode and, in my case, sometimes it crashes it.

Is there a workaround for it?

Thanks!

Best

Screenshot 2024-03-09 at 10 47 33 PM

Dragon-GCS commented 8 months ago

Same as you, I notice that when i update vscode 1.87, memory of process Code Help(Plugin) keep increasing when vscode lunched. Search pid by ps -ef|grep $pid, find the pylance (spwaned by Code Help(Plugin)), is eating my memory.

bschnurr commented 8 months ago

Thanks for logging an issue. Please see https://github.com/microsoft/pylance-release/blob/main/TROUBLESHOOTING.md#filing-an-issue For how to catching the log file.

If you can share a code sample or repo that reproduces the problem that would be best.

ale-dg commented 8 months ago

Hi @bschnurr ,

Thank you for your answer.

Not sure how to catch that log as it is not something I caught in VSCode but rather in the Activity Monitor from MacOS. As @Dragon-GCS mentioned, just catch the PID from the activity monitor and in the terminal shows which plug in is stealing so much memory and gets the machine hot as an oven, which in this case is Pylance.

I am not sure how often have you checked the main VSCode repo, but there have been a lot of issues reported where the he typing begins to lag (see https://github.com/microsoft/vscode-jupyter/issues/15351, https://github.com/microsoft/vscode/issues/206119, https://github.com/microsoft/vscode/issues/206865,). Even though some of them point to Jupyter, the one that seems to not "dump" memory is Pylance (Code Helper (Plugin)), which always stays over 1GB.

Let me know if I can help with something else.

Thank you again for your help.

Best,

heejaechang commented 8 months ago

if your goal is reducing memory footprint over having more functionality available, then you can try these to turn off some expensive features in settings.json

"python.analysis.indexing": false,
"python.analysis.typeCheckingMode": "off",
"python.analysis.enablePytestSupport": false,
"python.analysis.inlayHints.pytestParameters": false,
"python.analysis.inlayHints.callArgumentNames": "off",
"python.analysis.inlayHints.functionReturnTypes": false,
"python.analysis.inlayHints.variableTypes": false,
"python.analysis.autoImportCompletions": false,
"python.analysis.useLibraryCodeForTypes": false,
"editor.semanticHighlighting.enabled": false,

this will turn off us from digging into third party packages to figure out (infer) types when the packages don't support type annotations. usually that's the one that cause us to load a lot of third-party modules into memory.

since your main scenario is notebook, some of this info could still be fetched from kernel if you have run your notebooks.

bschnurr commented 8 months ago

also for the logs you need to copy the output window. "Select "View: Toggle Output" from the command palette (Ctrl+Shift+P on Windows/Linux, Command+Shift+P on macOS), then select "Python Language Server" in the dropdown on the right."

ale-dg commented 8 months ago

Hi @heejaechang ,

Thank you for your help. I understand I can turn all those assistances off for the sake of memory, but it does not explain the lagging nor the amount of hints it pops when using it. Yet again I could just uninstall it and call it quits.

In addition to me, I see others filing similar issues here, and even more dramatic problems than mine (https://github.com/microsoft/pylance-release/issues/5531)

This morning I switched back to VSCode 1.82.3, Jupyter v2023.8.1002501831, Pylance v2023.8.30 and it runs as smooth as it should. Pylance now "only" uses 712 MB (from 1.2GB last time I used the latest version) and uses only 19 threads (or 8.1% of the CPU, whereas before it was around 20%). I also see another user did this and it helped (https://github.com/microsoft/pylance-release/issues/5377). I have even disabled the automatic updates since now I am afraid something will break if it updates and I have been delayed for far too long troubleshooting this.

@Dragon-GCS would you be able to help us retrieving the logs as @bschnurr suggests so they can analyse the problem, please? I would gladly do it, but I have downgraded all the packages. 😜

Thanks

Best

heejaechang commented 8 months ago

@ale-dg, it's good to know that with VSCode 1.82.3, Jupyter v2023.8.1002501831, Pylance v2023.8.30, things work smoothly. Can you also tell us which python extension version you are using?

By the way, I am not saying we are not trying to address the performance issue. We are trying to figure it out, but so far, we are not getting enough information to reproduce it ourselves. This kind of performance issue is best addressed with a live reproduction so we can dig deeper. (We are fixing the ones that we could reproduce or investigate if users provide us with their project).

For your case as well, if you can provide us with the repository, such as your workspace (or a demo project that shows the symptoms) that we can open ourselves and investigate, then we might be able to help you much better.

heejaechang commented 8 months ago

by the way, this (https://github.com/microsoft/pylance-release/issues/5531) is different issue. that one is something is crashing (it could be us but not 100% sure from the info provided by the user) and creating core dump in the directory where pylance extension is installed.

if we are crashed, then we would expect core dump to be created in the executable folder (the folder vscode exe exists) not our extension folder where js files exist.

ale-dg commented 8 months ago

Hi @heejaechang

My apologies if I sounded too blunt, it wasn't my intention. Just trying to imply that if I turn off all the assistances, then there is not a case on holding to Pylance and just uninstalling it as soon as possible.

As of now, I can't share the notebook I am working on since it is a work-in-progress and it is not finished yet, and even so the file is quite heavy (38MB), so GitHub has its restrictions on it. What I can tell you is that the notebook has around 185 active coding cells and several markdowns (it's a deliverable, so the markdowns must be in there). As it's a data science project, naturally it has some DataFrames using pandas, some graphs using Matplotlib, Seaborn, Plotly and GeoPandas and I just began with some "light" machine learning using scikit-learn, everything with the latest releases as it is a new environment (see attached).

The Python extension version is v2024.2.1

Best

environment.yml.zip

heejaechang commented 8 months ago

hmm.. the enviroment.yml.zip link doesnt work. 404 error.

ale-dg commented 8 months ago

... weird... I just dropped it... now I've attached it... environment.yml.zip

Dragon-GCS commented 8 months ago

Hi @heejaechang ,

Thank you for your help. I understand I can turn all those assistances off for the sake of memory, but it does not explain the lagging nor the amount of hints it pops when using it. Yet again I could just uninstall it and call it quits.

In addition to me, I see others filing similar issues here, and even more dramatic problems than mine (#5531)

This morning I switched back to VSCode 1.82.3, Jupyter v2023.8.1002501831, Pylance v2023.8.30 and it runs as smooth as it should. Pylance now "only" uses 712 MB (from 1.2GB last time I used the latest version) and uses only 19 threads (or 8.1% of the CPU, whereas before it was around 20%). I also see another user did this and it helped (#5377). I have even disabled the automatic updates since now I am afraid something will break if it updates and I have been delayed for far too long troubleshooting this.

@Dragon-GCS would you be able to help us retrieving the logs as @bschnurr suggests so they can analyse the problem, please? I would gladly do it, but I have downgraded all the packages. 😜

Thanks

Best

Just saw it， I’ll try to reproduce this issue tomorrow on my Macbook

ale-dg commented 8 months ago

Hi @heejaechang @bschnurr

Just trying to keep all in the same loop. I opened another issue in VSCode related to the battery drainage and a lot of people jumped in it too. Another fellow loaded some info there (see https://github.com/microsoft/vscode-jupyter/issues/14459#issuecomment-1986677050) and does he does not have Pylance installed (according to him).

So, honestly now I am at odds on what is going on because apparently everything is draining memory and battery.

Best

ale-dg commented 8 months ago

Just for update... I tried to run the log of Pylance today with the latest updates as I was getting the logs for the Jupyter extension... and just a couple of things... just by opening VSCode, automatically logs 31,087 lines and I have no idea when does Pylance finishes. Then like 10,000 of them are just parsing and analysing cells . Even so, I attach it below, without running anything.

Pylance.log

Dragon-GCS commented 8 months ago

Bellowing test with a new vscode profile

Test Repo: https://github.com/Dragon-GCS/apscheduler-webui (or any workspace with a python file)

versions

Vscode 1.87.2

Extensions：(All of them is the latest with specified vscode version)

Python v2024.3.1
Pylance v2024.2.1
Copilot v1.41.0(latest is 0.141.0)
Copilot Chat 0.11.1(latest is 0.13.1)

Maybe problem is Copilot Chat, all works well when I update CopilotChat to latest version

Why I use this verion of copilot?

Because I have to different account to use copilit discussion

What I found

This copilot version working with vscode 1.87 will take a lot cpu and will cause vscode to crash , maybe because of bellowing logs
Copilot Chat won't take a lot memory, only with Pylance, the memory is increasing.
This suggestion icon is keep blinking

Logs

Copilot Chat (Repeating and not stoping until program broken down):

2024-03-14 10:38:41.496 [info] [auth] Logged in as Dragon-GCS
2024-03-14 10:38:41.496 [info] [chat] copilot token chat_enabled: true
2024-03-14 10:38:41.496 [info] [extension] Could not register explain command: Error: command 'github.copilot.interactiveEditor.explain' already exists
    at l.registerCommand (/Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/api/node/extensionHostProcess.js:150:183445)
    at Object.registerCommand (/Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/api/node/extensionHostProcess.js:160:20857)
    at o1 (/Users/dragon/.vscode/extensions/github.copilot-chat-0.11.1/dist/extension.js:1322:152)
    at khe (/Users/dragon/.vscode/extensions/github.copilot-chat-0.11.1/dist/extension.js:1322:5173)
    at xv.registerProviders (/Users/dragon/.vscode/extensions/github.copilot-chat-0.11.1/dist/extension.js:1324:7474)
    at xv.set activated [as activated] (/Users/dragon/.vscode/extensions/github.copilot-chat-0.11.1/dist/extension.js:1324:6716)
    at xv.set enabled [as enabled] (/Users/dragon/.vscode/extensions/github.copilot-chat-0.11.1/dist/extension.js:1324:6472)
    at Ho.<anonymous> (/Users/dragon/.vscode/extensions/github.copilot-chat-0.11.1/dist/extension.js:1326:178)
    at Ho.emit (node:events:526:35)
    at Ho.emit (/Users/dragon/.vscode/extensions/github.copilot-chat-0.11.1/dist/extension.js:184:22301)
    at H2 (/Users/dragon/.vscode/extensions/github.copilot-chat-0.11.1/dist/extension.js:188:12269)
    at kGe (/Users/dragon/.vscode/extensions/github.copilot-chat-0.11.1/dist/extension.js:1245:2854)
    at OGe (/Users/dragon/.vscode/extensions/github.copilot-chat-0.11.1/dist/extension.js:1245:3192)
    at PP.getCopilotToken (/Users/dragon/.vscode/extensions/github.copilot-chat-0.11.1/dist/extension.js:1245:4503)
    at xq.register (/Users/dragon/.vscode/extensions/github.copilot-chat-0.11.1/dist/extension.js:1258:1612)
2024-03-14 10:38:41.496 [info] [chat] Registration of chat agents failed:,Error: command 'github.copilot.interactiveSession.feedback' already exists
2024-03-14 10:38:41.574 [info] [auth] Logged in as Dragon-GCS
2024-03-14 10:38:41.574 [info] [chat] copilot token chat_enabled: true
2024-03-14 10:38:41.575 [info] [extension] Could not register explain command: Error: command 'github.copilot.interactiveEditor.explain' already exists
    at l.registerCommand (/Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/api/node/extensionHostProcess.js:150:183445)
    at Object.registerCommand (/Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/api/node/extensionHostProcess.js:160:20857)
    at o1 (/Users/dragon/.vscode/extensions/github.copilot-chat-0.11.1/dist/extension.js:1322:152)
    at khe (/Users/dragon/.vscode/extensions/github.copilot-chat-0.11.1/dist/extension.js:1322:5173)
    at xv.registerProviders (/Users/dragon/.vscode/extensions/github.copilot-chat-0.11.1/dist/extension.js:1324:7474)
    at xv.set activated [as activated] (/Users/dragon/.vscode/extensions/github.copilot-chat-0.11.1/dist/extension.js:1324:6716)
    at xv.set enabled [as enabled] (/Users/dragon/.vscode/extensions/github.copilot-chat-0.11.1/dist/extension.js:1324:6472)
    at Ho.<anonymous> (/Users/dragon/.vscode/extensions/github.copilot-chat-0.11.1/dist/extension.js:1326:178)
    at Ho.emit (node:events:526:35)
    at Ho.emit (/Users/dragon/.vscode/extensions/github.copilot-chat-0.11.1/dist/extension.js:184:22301)
    at H2 (/Users/dragon/.vscode/extensions/github.copilot-chat-0.11.1/dist/extension.js:188:12269)
    at kGe (/Users/dragon/.vscode/extensions/github.copilot-chat-0.11.1/dist/extension.js:1245:2854)
    at OGe (/Users/dragon/.vscode/extensions/github.copilot-chat-0.11.1/dist/extension.js:1245:3192)
    at PP.getCopilotToken (/Users/dragon/.vscode/extensions/github.copilot-chat-0.11.1/dist/extension.js:1245:4503)
    at xq.register (/Users/dragon/.vscode/extensions/github.copilot-chat-0.11.1/dist/extension.js:1258:1612)
2024-03-14 10:38:41.575 [info] [chat] Registration of chat agents failed:,Error: command 'github.copilot.interactiveSession.feedback' already exists
2024-03-14 10:38:41.634 [info] [auth] Logged in as Dragon-GCS
2024-03-14 10:38:41.635 [info] [chat] copilot token chat_enabled: true
2024-03-14 10:38:41.635 [info] [extension] Could not register explain command: Error: command 'github.copilot.interactiveEditor.explain' already exists
    at l.registerCommand (/Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/api/node/extensionHostProcess.js:150:183445)
    at Object.registerCommand (/Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/api/node/extensionHostProcess.js:160:20857)
    at o1 (/Users/dragon/.vscode/extensions/github.copilot-chat-0.11.1/dist/extension.js:1322:152)
    at khe (/Users/dragon/.vscode/extensions/github.copilot-chat-0.11.1/dist/extension.js:1322:5173)
    at xv.registerProviders (/Users/dragon/.vscode/extensions/github.copilot-chat-0.11.1/dist/extension.js:1324:7474)
    at xv.set activated [as activated] (/Users/dragon/.vscode/extensions/github.copilot-chat-0.11.1/dist/extension.js:1324:6716)
    at xv.set enabled [as enabled] (/Users/dragon/.vscode/extensions/github.copilot-chat-0.11.1/dist/extension.js:1324:6472)
    at Ho.<anonymous> (/Users/dragon/.vscode/extensions/github.copilot-chat-0.11.1/dist/extension.js:1326:178)
    at Ho.emit (node:events:526:35)
    at Ho.emit (/Users/dragon/.vscode/extensions/github.copilot-chat-0.11.1/dist/extension.js:184:22301)
    at H2 (/Users/dragon/.vscode/extensions/github.copilot-chat-0.11.1/dist/extension.js:188:12269)
    at kGe (/Users/dragon/.vscode/extensions/github.copilot-chat-0.11.1/dist/extension.js:1245:2854)
    at OGe (/Users/dragon/.vscode/extensions/github.copilot-chat-0.11.1/dist/extension.js:1245:3192)
    at PP.getCopilotToken (/Users/dragon/.vscode/extensions/github.copilot-chat-0.11.1/dist/extension.js:1245:4503)
    at xq.register (/Users/dragon/.vscode/extensions/github.copilot-chat-0.11.1/dist/extension.js:1258:1612)
2024-03-14 10:38:41.635 [info] [chat] Registration of chat agents failed:,Error: command 'github.copilot.interactiveSession.feedback' already exists

Pylance log Python Language Server.log

Maybe these can help figure out why pylance is taking up so much memory @bschnurr

@ale-dg Are you using this copilot too?

ale-dg commented 8 months ago

@Dragon-GCS nope, I have even just ran a "clean" VSCode with minimal extensions.

Best

bschnurr commented 8 months ago

@ale-dg I see this error at the bottom of the logs. "Workspace indexing has hit its upper limit: 2000 files"

did you try turning off indexing? see below

"python.analysis.indexing": false,
"python.analysis.typeCheckingMode": "off",
"python.analysis.enablePytestSupport": false,
"python.analysis.inlayHints.pytestParameters": false,
"python.analysis.inlayHints.callArgumentNames": "off",
"python.analysis.inlayHints.functionReturnTypes": false,
"python.analysis.inlayHints.variableTypes": false,
"python.analysis.autoImportCompletions": false,
"python.analysis.useLibraryCodeForTypes": false,
"editor.semanticHighlighting.enabled": false,

Are you using "openFileOnly" mode "python.analysis.diagnosticMode": "openFilesOnly",

You can all add directories to exclude "python.analysis.exclude": ["cdk.out"],

If none of that works try turning off type inference "python.analysis.useLibraryCodeForTypes": false,

ale-dg commented 8 months ago

@bschnurr nope... I just turned on the logging as suggested above 😅, let me try setting all of those and see what it logs

ale-dg commented 8 months ago

@bschnurr I have activated all the settings above. Let me know if it helps.

Best

Pylance-2.log

bschnurr commented 8 months ago

oh sorry. i mean you have to exclude any folders yourself, ["cdk.out"] was just an example.

 "python.analysis.exclude": [" your folder here"],

but does it seem fast now? after startup finishes the first analysis it should feel faster since more types are now cached.

ale-dg commented 8 months ago

oh sorry. i mean you have to exclude any folders yourself, ["cdk.out"] was just an example.
 "python.analysis.exclude": [" your folder here"],
but does it seem fast now? after startup finishes the first analysis it should feel faster since more types are now cached.

Well naturally if I turn off all the assistances it will feel faster, but when I turn them back on, everything begins to slowdown. @DonJayamanne opened an issue yesterday https://github.com/microsoft/vscode/issues/210528 for this. @heejaechang asked me to pull some cpuprofiles which I have uploaded there as well.

Would it be necessary to get the logs again? I'm asking for downgrading to the stable release again.

Best

JulianCBC commented 7 months ago

This is affecting me too with Pylance's server (~/.vscode-server/extensions/ms-python.vscode-pylance-2024.4.1/dist/server.bundle.js) using over 1.8GB of memory in a workspace with 26 folders, only 5 of which actually have Python scripts in them with a total count of 13 Python files.

This is quite frustrating as the VSC extensions used by the other part of my job - maintaining a very large PHP+React application - tend to scale their memory usage up and down as they do stuff, which means that during "busy" spikes, my WSL instance runs out of memory, causing VSC to slow to a crawl then crash.

This has been an issue for a very long time, and the fact that a FAQ entry has been written to detail how to work around out-of-memory issues strongly implies that the development team has basically given up on this problem.

Link for those who are curious: https://github.com/microsoft/pylance-release/blob/main/TROUBLESHOOTING.md#pylance-is-crashing

Could you please spend some development time exploring options to reduce the memory usage of this extension.

Some ideas:

Ignoring folders or projects that don't have python scripts in them - this will help users with multi-folder workspaces.
Pushing data to the frontend that doesn't need to be kept in the backend - this will help users with dev containers or WSL remotes
Purging data that isn't needed "now", e.g. analysing all cached data and balancing how much effort is required to reproduce it vs how much storage it uses - this will help everyone

I'm sorry if this is coming across as condescending but this is a very old issue and is seemingly being ignored because the majority of users of this extension seem to be throwing RAM at the issue rather than reporting it. It's doubly frustrating as other extension authors seem to be taking memory usage reasonably seriously.

Just to illustrate how other extensions are managing their memory usage, here are some stats from the VSC remote server for my workspace:

A community PHP language server: 1.3GB to manage 8 top-level folders with 66+ thousand PHP files
Official Terraform extension: 1GB to manage 3 top-level folders with hundreds of Terraform files in over a hundred configurations
The rest of the VSC remote instance including 24+ other extensions - this includes all the TypeScript / Javascript related extensions: ~400MB

kjersbry commented 3 months ago

I have the same problem. Pylance is consistently using 2.5 GB. My laptop is crashing frequently, and I haven't been able to find other causes so far. I'm using vscode with a docker container, and have multiple projects in my workspace. I have tried turning off the settings suggested in https://github.com/microsoft/pylance-release/issues/5614#issuecomment-1989619126.

jd-solanki commented 2 weeks ago

Pylance constantly taking up 1GB out of my RAM and sometimes it results in I can't load local LLMs to give them a try.

1GB for VSCode extension is too much.

Imagine opening 2-3 python projects at the same time, What that will do to the RAM 😢

JulianCBC commented 1 week ago

I'm frustratingly in the "throw RAM at the problem" camp now.

But still, on my large workspace running on a fairly beefy PC:

PyLance: 5 projects with ~13 Python files: 10.7% of RAM
Intelephence: 8 projects with 66+ thousand PHP files: 9.3% of RAM
ESLint: ~5 projects with thousands of TS and JS files: 4.3% of RAM

Or in other words: PyLance uses about twice as much RAM per-project as other extensions and over 2000 times as much RAM per-file.

My extensions are up-to-date.

Is anything actually being done about this?

rchiodo commented 1 week ago

The amount of python files you have isn't the thing that takes up memory, it's your imports. Internally the entire chain of imports is maintained in memory so that things like goto definition work instantly.

We've found that those imports can make a huge difference on the memory being used.

For example, all the docstrings defined for all the things in your import chain are stored in memory at the moment. For some packages, this can amount to MBs for a single file. Compound that over 1000s of imports chained together, and you end up with GBs of string data loaded.

At the moment speed is the priority though, not memory usage.

JulianCBC commented 1 week ago

@rchiodo while I very much appreciate your reply, it feels like you're blaming me for having a project that has "complex" imports instead of actually taking any responsibility for the problem: I don't really get to choose which things I import: if I want to make AWS API calls, I have to import boto3 and all it's dependencies. I get that you're explaining the reason why the memory usage is so high, but it still doesn't add up:

Looking at the largest of my projects - 8 Python files - I count 16 distinct modules imported, including modules from the standard library, and have 17 total external dependencies, i.e. ones which aren't part of the standard library. The virtualenv for that project has a total of 1206 python files in it.

So let's assume that all 5 projects are of equal size and round the number up: 1206 * 5 = 6030, so ~7000 Python files indexed.

In the PHP example above that "66+ thousand" includes all files in all external dependencies and has probably grown since my first post.

So this means that PyLance is using nearly 10x as much RAM per file as the PHP extension to provide fundamentally the same functionality.

Finally I must stress that this is memory usage of the language servers alone, I'm using Visual Studio Code with WSL so I'm not measuring the RAM usage of Visual Studio Code's UI itself.

So therefore I must question why all the docstrings and lookup data is in in RAM in WSL and not in the UI where it'll be used.

rchiodo commented 1 week ago

The architecture of Pylance/Pyright is such that nothing is in the UI. All of the logic is in a separate process. When you want to look at the docstring for a method, Pylance gets a textDocument/hover request. We then go and lookup the declaration of the method (meaning find the node where it actually resides in your import tree) and then return the docstring stored on the node. That docstring is stored in memory in that separate process (on the remote machine).

Generally, for these sorts of problems, we look at an example. We then do a memory analysis to see where all the memory is being used. If you can provide an example that seems egregious to you, it would make it easier for us to fix this.

We did one such analysis for stuff using AWS and found that we had 2 copies of docstrings in memory (that was a while ago, and that problem should have been fixed - had to do with the javascript substring function not reusing the same memory).

I think this issue would need more up votes for this problem to take priority over other work. It's working, it's just not ideal.

JulianCBC commented 1 week ago

The architecture of Pylance/Pyright is such that nothing is in the UI.

For the record: that's insane. Why the heck would this be architected in such a way that the data used by the UI is not in the UI process? And before you say "that's how the language server protocol works", I cannot see any scenario where a performant language server client would not be pre-fetching and caching this sort of data so the backend doesn't have to. I absolutely agree that performance of the server is important and work should be done there, but the client must also help it out by pre-fetching and caching data as needed.

I'm trying to get you a concrete example here, but I'm running into what appears to be the underlying problem here: if I open some random project in it's own window, either Pylance doesn't load, or it uses 200-300MB idle. Then, when I open a Python file, it uses about 300-500MB. My biggest Python project with the 16 modules imported uses ~500MB when all files are open in VSC.

In contrast, my big workspace with 27 projects uses over 1.6GB with no Python files open.

I suspect there's an issue here with workspaces.

ale-dg commented 1 week ago

I think this issue would need more up votes for this problem to take priority over other work. It's working, it's just not ideal.

Hi @rchiodo,

Thank you for the follow up. I guess we all agree in we are thankful for the effort you and the rest of the Pylance team invest in this extension. I am quoting your answer because I consider this does not invite us to report any bug we find if the first approach will be "if it works, don't touch it". We are reporting it because it is "working wrong", like a car with a flat tyre: it is rolling but it's limping.

I understand the priority is speed over memory, hence every import is "prefetched" and kept in memory. However I consider its a major oversight to not consider some of the most used scenarios of Python, which are Data Engineering and Data Science/Machine Learning/AI. These are tasks with heavy memory usage and using some of it for Pylance might be inconvenient for those tasks, especially for users with limited resources and/or servers with the same condition (i.e. running transactions in databases). Also, it's important to acknowledge VSCode uses a lot of resources just out of the box.

Best

microsoft / pylance-release