vscode takes up too much memory

[Otpvondoiats]

Environment

• OS and Version: Ubuntu 20.04.6 LTS
• VS Code Version: 1.78.1
• C/C++ Extension Version: v1.15.4
• If using SSH remote, specify OS of remote machine: None

Bug Summary and Steps to Reproduce

Bug Summary: Open a large project, such as aosp, the maximum memory usage exceeds 20G (computer 32G), VSCode settings are configured with a maximum 2G cache. The computer often freezes, how can I solve this problem

Configuration and Logs

none

Other Extensions

No response

Additional context

No response

[Stojek139808]

I've read an old issue just a moment ago while looking for a solution myself (yesterday got 16GB usage on a yocto project) https://github.com/microsoft/vscode-cpptools/issues/5227, and this comment mentions a possible solution (depending on your project)

[Otpvondoiats]

I've read an old issue just a moment ago while looking for a solution myself (yesterday got 16GB usage on a yocto project) #5227, and this comment mentions a possible solution (depending on your project) Thanks, but there will still be memory leaks when I deal with them in their way. Unless the C++ plugin is turned off

[sean-mcmanus]

Is there an open source project that repros this problem? Does setting the browse.path to empty ([]) solve the issue, i.e. is it caused by lots of files being added to the database? We don't know yet what is causing the high memory usage in cpptools. High memory usage is only expected if child cpptools-srv processes are taken into account.

[Otpvondoiats]

Is there an open source project that repros this problem? Does setting the browse.path to empty ([]) solve the issue, i.e. is it caused by lots of files being added to the database? We don't know yet what is causing the high memory usage in cpptools. High memory usage is only expected if child cpptools-srv processes are taken into account.

hi， https://github.com/apache/nuttx or google opensource aosp

[Otpvondoiats]

now, cpptool 14.5GiB

[Otpvondoiats]

cpp cache set:

plug list:

I also have this problem on Ubuntu 18.04. The memory has been with me for 4 or 5 years. I have changed many computers, only the plug-in list has not changed. what log do you need?I try to collect and provide it . @sean-mcmanus

[Otpvondoiats]

17.2GiB my Ubuntu crash.....

[sean-mcmanus]

So far we haven't been able to reproduce the issue.

[sean-mcmanus]

I think something external or specific to your machine is causing the issue.

Does the bug repro if you open another workspace folder with some test.cpp file? Are you able to determine what is special about the nuttx workspace that is causing the memory usage, such as copying the folder and removing/adding folders until the memory usage is not high?

[Otpvondoiats]

this my workspace size：

cpptool：

I will modify the file in the workspace, and no copy is involved. I would compile inside the workspace. I found that the memory usage is related to the number of texts in the workspace. If you open a small folder (only 40 ~ 50 files), this problem will not occur. If it is larger (tens of thousands) points, it will take up 2~5G

[Otpvondoiats]

I uninstalled the plugin again

[Otpvondoiats]

help me！！！

[Otpvondoiats]

@sean-mcmanus Hi，I found some clues. I found that he is caching the header files of my system. For example, my nuttx uses a header file. It will jump to the system by default. It is in the "ubuntu /usr" directory. How can I block it? Drop these things.

[sean-mcmanus]

Can you add folders you don't want processed to the C_Cpp.files.exclude setting?

[Otpvondoiats]

Can you add folders you don't want processed to the C_Cpp.files.exclude setting?

I added the path "/usr", but clicking on related functions in the code will still jump into the system's usr directory

[sean-mcmanus]

Yes, adding to the C_Cpp.files.exclude will still tag parse and go to def to files in the usr directory for any active files, but it should avoid tag parsing the files for non-open files, as can be verified by checking the logging after running the C/C++: Reset IntelliSense Database command. My thought was that it could reduce the memory usage (but we're not sure if that it's the cause or not).

[Otpvondoiats]

Yes, adding to the C_Cpp.files.exclude will still tag parse and go to def to files in the usr directory for any active files, but it should avoid tag parsing the files for non-open files, as can be verified by checking the logging after running the C/C++: Reset IntelliSense Database command. My thought was that it could reduce the memory usage (but we're not sure if that it's the cause or not).

After using it for two weeks, I found that this method does not work. Although it took effect at the time, it will still overflow in the past two weeks, but at a slower rate. Can this bug be fixed? It is very uncomfortable to use now. For example, I am looking at the linux kernel code recently, and I often use variables to jump. As the number of jumps increases, the memory usage increases (I set C_Cpp.intelliSenseCacheSize to 2048MB).

[Otpvondoiats]

Yes, adding to the C_Cpp.files.exclude will still tag parse and go to def to files in the usr directory for any active files, but it should avoid tag parsing the files for non-open files, as can be verified by checking the logging after running the C/C++: Reset IntelliSense Database command. My thought was that it could reduce the memory usage (but we're not sure if that it's the cause or not).

vscode version

版本: 1.80.0 提交: 660393deaaa6d1996740ff4880f1bad43768c814 日期: 2023-07-04T13:39:48.236Z 电子: 22.3.14 ElectronBuildId: 21893604 Chromium: 108.0.5359.215 Node.js: 16.17.1 V8: 10.8.168.25-electron.0 OS: Linux x64 5.15.0-76-generic

C/C++ plugs v1.16.3

[hiepkhuuMS]

@sean-mcmanus : what's the current status of this issue? I also repro it and memory jumps (and increasing) > 30GB till it runs out and everything fails, till I close VSCode.

Here's the original report I posted to VSCode which has more info about my repro and setup: https://github.com/microsoft/vscode/issues/198823

I have this repro currently, so feel free to let me know if you need live repro with remote windbgx or log for any further investigation.

[sean-mcmanus]

@hiepkhuuMS I don't think we've been able to repro this issue. It's possible your repro might be different, since all the previous repros were on Linux. If you set C_Cpp.loggingLevel to "Debug" and view the C/C++ logging that might show what is happening to accumulate that much memory, It's possible your directories are too large and you need to set C_Cpp.files.exclude. I think the Windows OS team has some "Wave" extension they use as a custom configuration provider so that might solve this as well.

[keithrausch]

I'm having a similar issue (cpptools consuming >20 GB of memory). I find that closing vs code will eventually kill the offending cpptools process, but it can take a couple minutes or more for that to happen. I'm attaching my workspace and extension list in case its useful

OS: Ubuntu 22.04.3 LTS C/C++ Extension Version: v1.18.5

Code Version:

Version: 1.81.1
Commit: 6c3e3dba23e8fadc360aed75ce363ba185c49794
Date: 2023-08-09T22:18:39.991Z
Electron: 22.3.18
ElectronBuildId: 22689846
Chromium: 108.0.5359.215
Node.js: 16.17.1
V8: 10.8.168.25-electron.0
OS: Linux x64 6.2.0-36-generic

I find that this typically takes a few hours to grow this much, but I'm having a hard time figuring out exactly what steps reproduce the issue. I will be following the instructions here and here to attach a debugger and enable logging, but this will take a few days due to the holidays. I expect to have something by next weak, as I encounter this issue nearly every day

screenshots of System Monitor:

Extensions:

[keithrausch]

Follow Up:

I opened vs code after posting my comment above, tweaked my launch.json and enabled Debugging output on the cpp extension per the linked instructions and then hit compile on a dirty build (I did not clear my build dir first). Around the same time, I opened a terminal and began a top -b | grep "cpptools" and loosely watched the output.

The project finished compiling in a couple minutes, and the Extension Debug output sat at 1.1GB and line 8389:

LSP: (invoked) cpptools/getInlayHints: file:///home/kpr/new_framework/libs/relative-gnc/associator/algo_associator.hpp (id: 54)

for a long time (roughly 26 minutes?). It then printed

Replaced placeholder file record

a few times and paused for a minute. Then sometime around the 26-30 minute mark (guessing according to the output of top), the memory usage crossed 1.2GB, it printed this log

  Discovering files: 3379780 file(s) processed
  74 file(s) removed from database
Done discovering files.
Parsing open files...
Populating include completion cache.
Parsing remaining files...

and then started vomiting files, and cpptools quickly rose to 1.4GB. In the time it took me to copy the output and start making these notes, it's risen to 2.3GB and its still pouring output.

It's now around 39 minutes in, and the memory usage is around 5GB. I'm catching glimpses of files in .trash and a lot of boost (lol). I'm also seeing it parse files in /mnt/windows which is far from expected behavior. It seems to be parsing my entire root directory including multiple different projects and OS partitions

At 46 minutes in, I see it parsing /home/kpr/.config/Code/User/History/63e209ea/Ti0R.cpp, and we're nearing 7GB. I think we have enough information at this point

I'm attaching the Debug output as well as the results of top.

top_cpptools.txt cpp_extension_debug_logging.txt

after an hour, we are now at 12GB, and I've now attached GDB to the cpptools process. Here is the output of -exec thread apply all bt

Thread 69 (LWP 51754 "cpptools"):
#0  0x00007fc004e52df5 in sccp ()
#1  0x00007fc004e5a3c0 in __timedwait_cp ()
#2  0x0000000000000000 in ?? ()

Thread 25 (LWP 6789 "cpptools"):
#0  0x00007fc004e52df5 in sccp ()
#1  0x00007fc004e5a3c0 in __timedwait_cp ()
#2  0x0000000000000000 in ?? ()

Thread 24 (LWP 6788 "cpptools"):
#0  0x00007fc004e52df5 in sccp ()
#1  0x00007fc004e5a3c0 in __timedwait_cp ()
#2  0x0000000000000000 in ?? ()

Thread 23 (LWP 6787 "cpptools"):
#0  0x00007fc004e52df5 in sccp ()
#1  0x00007fc004e5a3c0 in __timedwait_cp ()
#2  0x0000000000000000 in ?? ()

Thread 22 (LWP 6786 "cpptools"):
#0  0x00007fc004e52df5 in sccp ()
#1  0x00007fc004e5a3c0 in __timedwait_cp ()
#2  0x0000000000000000 in ?? ()

Thread 21 (LWP 6029 "cpptools"):
#0  0x00007fc004e52df5 in sccp ()
#1  0x00007fc004e5a3c0 in __timedwait_cp ()
#2  0x0000000000000000 in ?? ()

Thread 20 (LWP 6028 "cpptools"):
#0  0x00007fc004e52df5 in sccp ()
#1  0x00007fc004e483f4 in epoll_pwait ()
#2  0x0000000000000008 in ?? ()
#3  0x00007fc002fcfb64 in ?? ()
#4  0x0000000000000001 in ?? ()
#5  0xffffffffffffffff in ?? ()
#6  0x0000000000000000 in ?? ()

Thread 19 (LWP 6027 "cpptools"):
#0  0x00007fc004e52df5 in sccp ()
#1  0x00007fc004e5a3c0 in __timedwait_cp ()
#2  0x0000000000000000 in ?? ()

Thread 18 (LWP 6026 "cpptools"):
#0  0x00007fc004e52df5 in sccp ()
#1  0x00007fc004e480c6 in open64 ()
#2  0x0000000000000000 in ?? ()

Thread 17 (LWP 6025 "cpptools"):
#0  0x00007fc004e52df5 in sccp ()
#1  0x00007fc004e5a3c0 in __timedwait_cp ()
#2  0x0000000000000000 in ?? ()

Thread 16 (LWP 6024 "cpptools"):
#0  0x00007fc004e52df5 in sccp ()
#1  0x00007fc004e5a3c0 in __timedwait_cp ()
#2  0x0000000000000000 in ?? ()

Thread 15 (LWP 6023 "cpptools"):
#0  0x00007fc004e52df5 in sccp ()
#1  0x00007fc004e5a3c0 in __timedwait_cp ()
#2  0x0000000000000000 in ?? ()

Thread 14 (LWP 6022 "cpptools"):
#0  0x00007fc004e52df5 in sccp ()
#1  0x00007fc004e5a3c0 in __timedwait_cp ()
#2  0x0000000000000000 in ?? ()

Thread 13 (LWP 6021 "cpptools"):
#0  0x00007fc004e52df5 in sccp ()
#1  0x00007fc004e5a3c0 in __timedwait_cp ()
#2  0x0000000000000000 in ?? ()

Thread 12 (LWP 6020 "cpptools"):
#0  0x00007fc004e52df5 in sccp ()
#1  0x00007fc004e5a3c0 in __timedwait_cp ()
#2  0x0000000000000000 in ?? ()

Thread 11 (LWP 6019 "cpptools"):
#0  0x00007fc004e52df5 in sccp ()
#1  0x00007fc004e5a3c0 in __timedwait_cp ()
#2  0x0000000000000000 in ?? ()

Thread 10 (LWP 6018 "cpptools"):
#0  0x00007fc004e52df5 in sccp ()
#1  0x00007fc004e5a3c0 in __timedwait_cp ()
#2  0x0000000000000000 in ?? ()

Thread 9 (LWP 6017 "cpptools"):
#0  0x00007fc004e52df5 in sccp ()
#1  0x00007fc004e5a3c0 in __timedwait_cp ()
#2  0x0000000000000000 in ?? ()

Thread 8 (LWP 6016 "cpptools"):
#0  0x00007fc004e52df5 in sccp ()
#1  0x00007fc004e5a3c0 in __timedwait_cp ()
#2  0x0000000000000000 in ?? ()

Thread 7 (LWP 6015 "cpptools"):
#0  0x00007fc004e52df5 in sccp ()
#1  0x00007fc004e5a3c0 in __timedwait_cp ()
#2  0x0000000000000000 in ?? ()

Thread 6 (LWP 6014 "cpptools"):
#0  0x00007fc004e52df5 in sccp ()
#1  0x00007fc004e5a3c0 in __timedwait_cp ()
#2  0x0000000000000000 in ?? ()

Thread 5 (LWP 6013 "cpptools"):
#0  0x00007fc004e52df5 in sccp ()
#1  0x00007fc004e5a3c0 in __timedwait_cp ()
#2  0x0000000000000000 in ?? ()

Thread 4 (LWP 6012 "cpptools"):
#0  0x00007fc004e52df5 in sccp ()
#1  0x00007fc004e5a3c0 in __timedwait_cp ()
#2  0x0000000000000000 in ?? ()

Thread 3 (LWP 6011 "cpptools"):
#0  0x00007fc004e52df5 in sccp ()
#1  0x00007fc004e5a3c0 in __timedwait_cp ()
#2  0x0000000000000000 in ?? ()

Thread 2 (LWP 6010 "cpptools"):
#0  0x00007fc004e52df5 in sccp ()
#1  0x00007fc004e483f4 in epoll_pwait ()
#2  0x0000000000000008 in ?? ()
#3  0x00007fc00593c950 in ?? ()
#4  0x0000000000000001 in ?? ()
#5  0xffffffffffffffff in ?? ()
#6  0x0000000000000000 in ?? ()

Thread 1 (LWP 6008 "cpptools"):
#0  0x00007fc004e4e0a9 in __stdio_read ()
#1  0x00007ffcc008831f in ?? ()
#2  0x0000000000000000 in ?? ()

[Otpvondoiats]

@sean-mcmanus

版本: 1.86.2 提交: 903b1e9d8990623e3d7da1df3d33db3e42d80eda 日期: 2024-02-13T19:41:37.860Z Electron: 27.2.3 ElectronBuildId: 26908389 Chromium: 118.0.5993.159 Node.js: 18.17.1 V8: 11.8.172.18-electron.0 OS: Linux x64 5.15.0-94-generic

There will still be this problem

[tshcherban]

Reproducible on both 1.19.9 and 1.20.0. On editing cpp files Code firing multiple processes (cpptools-src), each taking 8Gb commit size (working set remains low) until VSCode crashes because OutOfMemory. The sources im editing are an Arduino project, which uses ArduinoJson. Maybe this related?

[Colengms]

Hi @tshcherban . You may be repro'ing a different issue. Most of the comments in this issue are now months old. I would suggest opening a new issue and providing the information requested in our new issue template, including your current settings/configuration and debug log output.

In this thread, I see logs only from @keithrausch . @keithrausch, you appear to have repro'ed an issue in which the C/C++ Extension is attempting to tag-parse your entire file system (including /proc, and all /mnt points). It's possible this could be by-design if you've specified the root of your file system in your configuration as a include path or browse path, which is generally not desirable. We have fixed an issue recently in 1.19.9 that might have addressed a rare cause of this particular scenario. I would be curious if you're still seeing that issue with recent versions.

[sean-mcmanus]

@tshcherban Your issue sounds like the Win7 issue with virtual memory in thread https://github.com/microsoft/vscode-cpptools/issues/9144 . This issue is tracking a Linux/Mac issue.

[keithrausch]

@sean-mcmanus I'm currently running v1.19.9 and am still very much having this issue. My extension does not have any default include or browse paths specified (I checked these fields in settings: C_Cpp › Default: Include Path and C_Cpp › Default › Browse: Path)

Upon closer inspection, I do have a rogue target_include_directories call in my CMake that is including 1 directory higher than it should (it's including my entire user directory). I have a hard time believing that there's a sym link from my user directory to root, but I can't be certain. I'll fix this and report back.

If this doesn't fix my problem, do you have any suggestions or debug prints that I can use to figure out where my problem might be?

[sean-mcmanus]

@keithrausch Yeah, you can set C_Cpp.loggingLevel to "Debug" and look at the C/C++ logging for unexpected references to paths outside your workspace folder or paths associated with your compiler's system paths.

[Otpvondoiats]

debug

Now open android aosp and it takes up 29G.

[keithrausch]

As far as I can tell, I haven't had this bug in a while. I'm going to chalk it up to a bad include path and use error. Thank you for your time!

[Otpvondoiats]

As far as I can tell, I haven't had this bug in a while. I'm going to chalk it up to a bad include path and use error. Thank you for your time! @keithrausch Yes, I also think it is a problem of path inclusion, especially on the Linux platform. However, the exclusion path set in the configuration path of the project and vsode software will not take effect, and the code jump will still jump to the system's /usr/include. I have tried many methods and have not been able to solve it.

[Otpvondoiats]

@keithrausch Or to put it another way, vscode provides methods to specify include paths or shield system paths. I googled many methods, but none of them worked.

[Neywiny]

I'm seeing this as well but a lot worse. Mine takes 44GB. The last thing I saw was: "Parsing open files | Populating include completion cache." I set the max memory to 2048 but that seems to have not mattered for this step. I have early OOM so eventually my computer should come back but right now it's frozen so I can't check logs.

[Otpvondoiats]

I'm seeing this as well but a lot worse. Mine takes 44GB. The last thing I saw was: "Parsing open files | Populating include completion cache." I set the max memory to 2048 but that seems to have not mattered for this step. I have early OOM so eventually my computer should come back but right now it's frozen so I can't check logs.

It's hard to use now,

[bobbrow]

@Neywiny if you are able to attach a debugger and share the threads' call stacks when you think that it's misbehaving, we can take a look. Otherwise, we don't have a lot to go on and we don't observe this in the workspaces we've been testing with. Some instructions can be found here: Attaching debugger to cpptools

@Otpvondoiats, are you still working with the android aosp? I'll see if I can reproduce this.

[Otpvondoiats]

@Neywiny if you are able to attach a debugger and share the threads' call stacks when you think that it's misbehaving, we can take a look. Otherwise, we don't have a lot to go on and we don't observe this in the workspaces we've been testing with. Some instructions can be found here: Attaching debugger to cpptools

@Otpvondoiats, are you still working with the android aosp? I'll see if I can reproduce this.

Yes, the plugin is working properly. This happens on both Ubuntu 2004 and 2204.

[bobbrow]

Yes, the plugin is working properly. This happens on both Ubuntu 2004 and 2204.

I'm confused, I thought you said it was using too much memory. Is there still a problem we need to investigate for you?

[Otpvondoiats]

Yes, the plugin is working properly. This happens on both Ubuntu 2004 and 2204.

I'm confused, I thought you said it was using too much memory. Is there still a problem we need to investigate for you?

Sorry, I expressed some differences. The current plug-in always takes up a lot of memory, aosp projects or nuttx code. What I mean by normal is that there is no abnormality in the plug-in environment.

[bobbrow]

Thank you. I'm taking a look at the android workspace and although I'm not sure I got it cloned properly, there are still a lot of files and I'm seeing a 11GB database with 11GB of resident memory which seems like a close enough repro.

To start, one thing that will help reduce the size is to tell the extension to exclude folders that don't need to be scanned, such as the .repo folder in the android workspace. I'll be taking a deeper look over the next while to see what else may be contributing to the size.

For example, add this to your settings:

  "C_Cpp.files.exclude": {
    "**/.repo": true
  }

However, for .repo specifically you may also consider using the files.exclude setting which will stop VS Code from scanning it too.

[Neywiny]

@bobbrow This is on an airgapped machine, so even if I did get the debugger attached, I wouldn't be able to send anything to you. During workspace parsing, millions of files are identified. In my home directory, baobab showed something like 20-30GB of data (each) being stored for a few workspaces.

I can give some context but because of sensitivity I can't give the workspace itself. It's a custom buildroot git repo that has a decent amount of history. I think I have 2-3 full builds (of different configurations/versions) on the computer at any given time. I haven't tried excluding .git or anything. I haven't tried a clean buildroot pull on a networked machine with cpptools to try and make a reproducible example, but if I can get more availability I will.

[Otpvondoiats]

Thank you. I'm taking a look at the android workspace and although I'm not sure I got it cloned properly, there are still a lot of files and I'm seeing a 11GB database with 11GB of resident memory which seems like a close enough repro.

To start, one thing that will help reduce the size is to tell the extension to exclude folders that don't need to be scanned, such as the .repo folder in the android workspace. I'll be taking a deeper look over the next while to see what else may be contributing to the size.

For example, add this to your settings:

  "C_Cpp.files.exclude": {
    "**/.repo": true
  }

However, for .repo specifically you may also consider using the files.exclude setting which will stop VS Code from scanning it too.

I have tried this, and I also added the system root directory ("/"), and it still increases. For the same code repository, I used clangd and found that its memory usage was around 11G. cpptool would stabilize at more than 10G in the short term, and it would increase wildly over time. I don't know much about the debugger method of the vscode plug-in, so I can't provide you with logs.