IntelliSense using too much memory and resetting pch

[sean-mcmanus]

Ok, got it.

Your PCH some logic has issues. I have 2 files PawnTank.h and PawnTank.cpp. If in PawnTank.ccp when I do a look up (hover with with left mouse button held down) cpptools.srv goes up as high as 2GB Real Mem then generates a 1.7GB PCH cache file for PawnTank.h. This takes about 50 seconds. If I close PawnTank.cpp, then reopen it, the whole process takes another 60 seconds as PawnTank.h PCH is regenerated unnecessarily yet again.

You're building a huge in memory cache structure, then dumping it when the file is closed. I recommend some sort've memory mapped DB that is local plus better cache invalidation logic.

The UE4 tutorial "Toon Tanks" is here https://gitlab.com/GameDevTV/UnrealCourse/ToonTanks. Install UE4, clone "Toon Tanks", build and step around, you'll see.

I didn't want this to be a career move so I'm going to live with what I've got.

I switched C_Cpp Tag Parser and for the time being, it works like a charm.

Originally posted by @magwa101 in https://github.com/microsoft/vscode-cpptools/issues/7143#issuecomment-796247121

[sean-mcmanus]

@magwa101 The pch file is memory mapped and it's not supposed to be recreated unless the header file is changed. It sounds like there's some bug or memory issue going on.

[v-ericawu]

@magwa101, sorry for we failed to repro your issue, could you help to confirm 3 questions?

1. Could you help to confirm whether our repro steps is correct or not?

   • Install UE4
   • Clone "Toon Tanks": https://gitlab.com/GameDevTV/UnrealCourse/ToonTanks
   • Build the project
   • After build completed, hover with with left mouse button held down, check cpptools.srv goes up as high as 2GB Real Mem then generates a 1.7GB PCH cache file for PawnTank.h

2. Once our steps is correct, could you help to confirm whether we missed anything when clone "Toon Tanks"?

• As we can't find some files which listed in "includePath" of "c_cpp_properties.json":

• Currently, we failed to build project with error: Windows: Mac:

3. Could you help to confirm your OS is Windows or Mac? We tried to build the project on Mac, while we are not sure how to set the correct platform? (For windows, the value is "Win32", "Win64")

[magwa101]

Mac OSX: 10.15.7 Build target can be "ToonTanksEditor Mac Development Build". This will avoid the error about the type of the "engine distribution".

After build, poke around the project code to provoke intellisense into "indexing". Watch for reported issues in memory growth related to cached file sizes.

Thanks your diligence.

[v-ericawu]

@magwa101 We are trying to build the project on Mac and there is still 3 issues need confirm with you:

1. Just as we previous mentioned on Windows, there is some paths do not exist which listed in "includePath" of "c_cpp_properties.json", could you help to confirm how to fix them or could we skip that directly?

2. We updated the build target args as below, while still build failed, could you help to confirm whether we used the correct args? args status on Mac:

3. Some header files error:

Can you help to confirm that the above error is caused by a non-existent path on Mac OS?

[boocs]

I believe the Debug config is for the full UE4 source. Use DebugGame if you don't have the full source version of UE4.

Usually I use Development config for nominal builds and DebugGame for debugging.

[magwa101]

Yes, use the Development Bulid that's fine, or probably, DebugGame is probably fine too.

However my understanding is the Intellisense has it's own 'compilation' phase that exists outside of the particular programs build system. So, do you need to build the game? Anyway, if you build it successfully, double click on toontank.uproject and have a go with the tanks.

[v-ericawu]

@magwa101 Could you help to confirm two questions after build successfully? 1.Double-click ToonTanks.uproject, and the code is not colored： Do you have a similar problem, if not, can you help to confirm the cause of the problem? 2.Open PawnTank.cpp and hover with left mouse button held down, The cpptools.srv memory will not reach 2G: If the step is incorrect, can you share the detailed steps?

[magwa101]

It was growing when trying to generate precompiled header files (PCH).

After you double click on the uproject file you will see a compile button inside the UE interface, in the toolbar near the top. Go ahead and hit compile, everything should build from within UE. This will generate all their complex header files that are then precompiled. Then go into PawnTank.cpp and bounce around to different functions, in particular, F12 "Go to definition" within the UE source code.

BTW, do you consider using clang with intellisense?

[sean-mcmanus]

@magwa101 What do you mean by "using clang with IntelliSense"? Do you mean using clang as a parser for IntelliSense instead of EDG?

[magwa101]

Clangd like this VS plugin.

[sean-mcmanus]

I think we tried several years ago and it was slower. Not sure if that's still the case. I'm also not sure if clangd supports as wide a variety of compilation modes as we support.

[v-ericawu]

@magwa101 We didn't repro your issue according your steps:

And If use 'clangd', IntelliSense will be disabled:

[magwa101]

I tried again with 1.3.0-insiders2 and it creates these huge never completing processes: [image: Screen Shot 2021-04-01 at 3.05.57 PM.png]

I set my includes to .../ToonTanks/ and /Epic Games/UE_4.26/Engine/

C is C17 C++ is C++17

Again, it's CPU pegged and making no suggestions, I'm going back to "tags parser".

On Tue, Mar 30, 2021 at 9:36 PM v-ericawu @.***> wrote:

@magwa101 https://github.com/magwa101 We didn't repro your issue according your steps: [image: UEProject_NotRepro] https://user-images.githubusercontent.com/78774400/113090564-a7fc4700-921c-11eb-81fe-21b9213f6d73.gif

And If use 'clangd', IntelliSense will be disabled: [image: clangd_disable] https://user-images.githubusercontent.com/78774400/113090575-b185af00-921c-11eb-94f2-85d160d73c19.gif

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/microsoft/vscode-cpptools/issues/7147#issuecomment-810753896, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAGVKW7N27DS2MB6LUDHM63TGKRGHANCNFSM4Y7BLTPQ .

[boocs]

Don't use the VSCode 'includePath' setting anymore for 4.26.+

4.26.1 uses compile commands and response files to determine what includes to use. You should reset your UE4 project to get rid of any c_cpp_properties changes.

Whenever you add a new .h/.cpp file you need to reset your Unreal project if you start getting Intellisense errors. Unreal should add the new files to the compile commands file. The new source/header files should have the correct includes set for Intellisense.

I made an UE/VSCode config fix extension if you want to try it (only tested with cpptools extension): If you use the extension you don't have to reset your project to add new files to the compile commands. Just reopen VSCode and they will be added by the extension. https://gist.github.com/boocs/f63a4878156295b6e854cac68672f305

[v-ericawu]

@magwa101 The CPU usage is also normal when using 1.3.0-insiders2:

[magwa101]

I opened ToonTank.uproject and the json is not color coded.

boocs implies that includes with "/" is a problem. I regenerated the project in UE. It changed C to c11 and C++ to c++98. Includes with "../" were removed. So far it's working much better! I will try it out today. Thanks boocs!

Prior includes to (likely) provoke the CPU/memory issue:

• UE_4.26/Engine/**
• ToonTanks/**

This is with intellisense 1.2.2, Mac 10.15.7, vsc 1.55.0 UE4.26.1

On Wed, Mar 24, 2021 at 7:54 PM v-ericawu @.***> wrote:

@magwa101 https://github.com/magwa101 Could you help to confirm two questions after build successfully? 1.Double-click ToonTanks.uproject, and the code is not colored： [image: vscode9] https://user-images.githubusercontent.com/78774400/112409710-f90bc700-8d54-11eb-8fe8-ff51d24bf1bf.PNG Do you have a similar problem, if not, can you help to confirm the cause of the problem? 2.Open PawnTank.cpp and hover with left mouse button held down, The cpptools.srv memory will not reach 2G: [image: vscode8] https://user-images.githubusercontent.com/78774400/112410224-d332f200-8d55-11eb-94be-bb542a1c4b82.PNG If the step is incorrect, can you share the detailed steps?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/microsoft/vscode-cpptools/issues/7147#issuecomment-806325073, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAGVKW6WV7XASI327NFMHI3TFKQV3ANCNFSM4Y7BLTPQ .

[magwa101]

Ok, I spoke too soon, unfortunately I'm a VSC n00b and had a "tags parser" in my global settings.json that probably over rode my "intellisense" (via default) setting in my workspace. I don't understand the hierarchy of json file includes yet and have not seen it explained in the docs.

In any case, memory is climbing, CPU is very high and lookups are not resolving, it's been 10 minutes now. So, yes, I still see a problem: [image: Screen Shot 2021-04-02 at 6.16.21 AM.png] [image: Screen Shot 2021-04-02 at 5.55.17 AM.png]

On Fri, Apr 2, 2021 at 5:44 AM Jeremy Daw @.***> wrote:

I opened ToonTank.uproject and the json is not color coded.

boocs implies that includes with "/" is a problem. I regenerated the project in UE. It changed C to c11 and C++ to c++98. Includes with "../" were removed. So far it's working much better! I will try it out today. Thanks boocs!

Prior includes to (likely) provoke the CPU/memory issue:

• UE_4.26/Engine/**
• ToonTanks/**

This is with intellisense 1.2.2, Mac 10.15.7, vsc 1.55.0 UE4.26.1

On Wed, Mar 24, 2021 at 7:54 PM v-ericawu @.***> wrote:

@magwa101 https://github.com/magwa101 Could you help to confirm two questions after build successfully? 1.Double-click ToonTanks.uproject, and the code is not colored： [image: vscode9] https://user-images.githubusercontent.com/78774400/112409710-f90bc700-8d54-11eb-8fe8-ff51d24bf1bf.PNG Do you have a similar problem, if not, can you help to confirm the cause of the problem? 2.Open PawnTank.cpp and hover with left mouse button held down, The cpptools.srv memory will not reach 2G: [image: vscode8] https://user-images.githubusercontent.com/78774400/112410224-d332f200-8d55-11eb-94be-bb542a1c4b82.PNG If the step is incorrect, can you share the detailed steps?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/microsoft/vscode-cpptools/issues/7147#issuecomment-806325073, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAGVKW6WV7XASI327NFMHI3TFKQV3ANCNFSM4Y7BLTPQ .

[magwa101]

I spent about 3 hours on this, because you know, I want a good C++ parser for the long term.

My conclusion, in my mac environment, is to run "tag parser" and in my workspace settings.json put /engine/ and /, reset the intellisense database, switch to tag parser. That's it, it's not the best, but it works, and it's fast.

@boocs Boocs thanks for the suggestions but I'm not installing anything that does auto "fixes". Also, the "rsp" files include all the includes in UE4 anyway, so UE4/engine/** is the same thing.

@v-ericawu great tenacity, please tell me if you can reproduce.

I'm honestly not sure who is MS employee, who is not, but I don't see this issue as a priority to MS. It's a pretty obvious problem and people all over the webs do complain aboout VSC and UE on Mac.

I have seen people report great outcomes with clang as the C++ parser but the configuration is too much "do this/then that/then the other" so I'm waiting and hoping it gets packaged up. If I go back onto this it will be via clangd extension: https://marketplace.visualstudio.com/items?itemName=llvm-vs-code-extensions.vscode-clangd

I'm still checking this thread, occassionally.

[sean-mcmanus]

@magwa101 We're not able to repro the issue. @v-ericawu is helping us investigate repros for some issues. Maybe it requires a particular MacOS version or XCode to be installed?

[magwa101]

MacOS 10.15.7 Xcode 12.4

clang --version 12.0.0

On Fri, Apr 2, 2021 at 12:52 PM Sean McManus @.***> wrote:

@magwa101 https://github.com/magwa101 We're not able to repro the issue. @v-ericawu https://github.com/v-ericawu is helping us investigate repros for some issues. Maybe it requires a particular MacOS version or XCode to be installed?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/microsoft/vscode-cpptools/issues/7147#issuecomment-812685443, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAGVKW272RFNJZB6BEVUQODTGYOALANCNFSM4Y7BLTPQ .

[boocs]

@magwa101

By default UE4 will create a ~1.6 GB Tag Parser cache. The UE4 workspace needs to set browse.path to a blank array [] to fix this. You also need to make sure limitSymbolsToIncludedHeaders is set to true in the main workspace. Do this will reduce the Tag Parser cache by 0.6 GB. I do fix this in my extension but read the bottom line about a possible extension bug.

By default VSCode will delete/create a ~1 GB Intellisense cache every time you type a line in your header file. Source files are a little better but it will also do it if you create a new function. This is because the default includes in UE4 headers are huge and how VSCode recompiles Intellisense.

You can try setting C_Cpp: Intelli Sense Cache Size to 0 in your global user settings to prevent this from happening.

Maybe these fixes will work to fix your problem.

Also I found that there maybe a bug in my extension. I have clang using a default cppStandard of c++14 but I think it should be using gnu++14. So I'd wait till I release a fix.

[magwa101]

I saw the Tag Parser cache reach 1.2GB so I can confirm your estimate. That's not of much concern really but great advice on size reduction.

You hit the nail on the head(er) file. Edits of my headers caused mass recompiles, this rendered it unusable in my estimation. This is the problem. Plain and simple.

I'd like to see an easier install/config of VSIX clangd, I think this is a better path.

On Fri, Apr 2, 2021 at 2:33 PM boocs @.***> wrote:

@magwa101 https://github.com/magwa101

By default UE4 will create a ~1.6 GB Tag Parser cache. The UE4 workspace needs to set browse.path to a blank array [] to fix this. You also need to make sure limitSymbolsToIncludedHeaders is set to true in the main workspace. Do this will reduce the Tag Parser cache by 0.6 GB.

By default VSCode will delete/create a ~1 GB Intellisense cache every time you type a line in your header file. Source files are a little better but it will also do it if you create a new function. This is because the default includes in UE4 headers are huge and how VSCode recompiles Intellisense.

You can try setting C_Cpp: Intelli Sense Cache Size to 0 in your global user settings to prevent this from happening.

Maybe these fixes will work to fix your problem.

Also I found that there maybe a bug in my extension. I have clang using a default cppStandard of c++14 but I think it should be using gnu++14. So I'd wait till I release a fix.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/microsoft/vscode-cpptools/issues/7147#issuecomment-812725076, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAGVKW2QUGDZCEBNBUEHHALTGYZ3NANCNFSM4Y7BLTPQ .

[boocs]

@magwa101 I forgot to mention, setting Intellisense cache size to 0 will only prevent the cache from being create on your hard drive. It'll still need recompile which will still take time.

I'm using the cl.exe compiler. With the latest cl.exe compiler it usually takes ~15 seconds to recompile Intellisense in a UE4 project. A few months ago it was taking 30-40 seconds.

15 seconds still makes it hard to use Intellisense in header files but at least it's getting better.

Source files seem decent enough since Intellisense won't/shouldn't recompile when you're typing inside a function.

[v-ericawu]

@magwa101 I opened ToonTank.uproject and the json still has no color coding when changing C/C++. and Could you help to verify a problem? "../" has been deleted" means to delete the "./" in which file?

[xudxud]

@magwa101 I forgot to mention, setting Intellisense cache size to 0 will only prevent the cache from being create on your hard drive. It'll still need recompile which will still take time.

I'm using the cl.exe compiler. With the latest cl.exe compiler it usually takes ~15 seconds to recompile Intellisense in a UE4 project. A few months ago it was taking 30-40 seconds.

15 seconds still makes it hard to use Intellisense in header files but at least it's getting better.

Source files seem decent enough since Intellisense won't/shouldn't recompile when you're typing inside a function.

I'm new to UE and C++.

Every time I change the header file it'll take about half minute to reset the intellisense cache file (.ipch file nearly 3GB). I thought it was not normal before, it seems I was wrong... Thanks for your answer!

By the way I've installed your extension before even hoped it could solve the problem lol