search

Andersbakken / rtags

A client/server indexer for c/c++/objc[++] with integration for Emacs based on clang.
http://www.rtags.net
GNU General Public License v3.0
1.83k stars 253 forks source link

High Memory usage and failing to Fork #1263

Open AfterThunk opened 5 years ago

AfterThunk commented 5 years ago

I'm trying to use RTags to index a fairly complex project (UnrealEngine 4). However, when I do this it seems to make it through a very small subset of files, and then everything else immediately fails.

My guess is that there's some memory leak that is causing rdm to grow. Processes are forked without sharing memory (maybe they can't?) and some limit is reached. After that limit (~50% total system memory), future forks fail and subsequent attempts to index files fail.

Reproduce

Expected behavior Indexing completes successfully (albeit taking a long time). FWIW, I'm not sure if this is a new bug or a regression.

Actual Behavior rdm quickly eats up a ton of memory (~50% of my total memory). After that, forking fails and so does indexing.

[100%] 8450/8469 01:38:00 ~/Coding/Projects/C++/UE4/UnrealEngine/Engine/Plugins/Tests/EditorTests/Source/EditorTests/Private/UnrealEd/EditorTestsUtilityLibrary.cpp indexing crashed.
Couldn't start rp /home/widmo/Coding/Projects/C++/RTags/rtags/bin/rp Fork failed
[100%] 8451/8469 01:38:00 ~/Coding/Projects/C++/UE4/UnrealEngine/Engine/Plugins/Tests/EditorTests/Source/EditorTests/Private/UnrealEd/EditorViewportFlagsTest.cpp indexing crashed.
Couldn't start rp /home/widmo/Coding/Projects/C++/RTags/rtags/bin/rp Fork failed
[100%] 8452/8469 01:38:00 ~/Coding/Projects/C++/UE4/UnrealEngine/Engine/Plugins/Tests/EditorTests/Source/EditorTests/Private/UnrealEd/FbxAutomationTests.cpp indexing crashed.
Couldn't start rp /home/widmo/Coding/Projects/C++/RTags/rtags/bin/rp Fork failed
[100%] 8453/8469 01:38:00 ~/Coding/Projects/C++/UE4/UnrealEngine/Engine/Plugins/Tests/EditorTests/Source/EditorTests/Private/UnrealEd/FeaturePackAutomationTests.cpp indexing crashed.
Couldn't start rp /home/widmo/Coding/Projects/C++/RTags/rtags/bin/rp Fork failed
[100%] 8454/8469 01:38:00 ~/Coding/Projects/C++/UE4/UnrealEngine/Engine/Plugins/Tests/EditorTests/Source/EditorTests/Private/UnrealEd/GeometryTests.cpp indexing crashed.
Couldn't start rp /home/widmo/Coding/Projects/C++/RTags/rtags/bin/rp Fork failed
[100%] 8455/8469 01:38:00 ~/Coding/Projects/C++/UE4/UnrealEngine/Engine/Plugins/Tests/EditorTests/Source/EditorTests/Private/UnrealEd/LightingTests.cpp indexing crashed.
Couldn't start rp /home/widmo/Coding/Projects/C++/RTags/rtags/bin/rp Fork failed
[100%] 8456/8469 01:38:00 ~/Coding/Projects/C++/UE4/UnrealEngine/Engine/Plugins/Tests/EditorTests/Source/EditorTests/Private/UnrealEd/MeshDescriptionTests.cpp indexing crashed.
Couldn't start rp /home/widmo/Coding/Projects/C++/RTags/rtags/bin/rp Fork failed
[100%] 8457/8469 01:38:00 ~/Coding/Projects/C++/UE4/UnrealEngine/Engine/Plugins/Tests/FbxAutomationTestBuilder/Source/FbxAutomationTestBuilder/Private/FbxAutomationBuilder.cpp indexing crashed.
Couldn't start rp /home/widmo/Coding/Projects/C++/RTags/rtags/bin/rp Fork failed
[100%] 8458/8469 01:38:00 ~/Coding/Projects/C++/UE4/UnrealEngine/Engine/Plugins/Tests/FbxAutomationTestBuilder/Source/FbxAutomationTestBuilder/Private/FbxAutomationBuilderModule.cpp indexing crashed.
Couldn't start rp /home/widmo/Coding/Projects/C++/RTags/rtags/bin/rp Fork failed
[100%] 8459/8469 01:38:00 ~/Coding/Projects/C++/UE4/UnrealEngine/Engine/Plugins/Tests/FbxAutomationTestBuilder/Source/FbxAutomationTestBuilder/Private/FbxAutomationBuilderStyle.cpp indexing crashed.
Couldn't start rp /home/widmo/Coding/Projects/C++/RTags/rtags/bin/rp Fork failed
[100%] 8460/8469 01:38:00 ~/Coding/Projects/C++/UE4/UnrealEngine/Engine/Plugins/Tests/FunctionalTestingEditor/Source/Private/AssetTypeActions_GroundTruthData.cpp indexing crashed.
Couldn't start rp /home/widmo/Coding/Projects/C++/RTags/rtags/bin/rp Fork failed
[100%] 8461/8469 01:38:00 ~/Coding/Projects/C++/UE4/UnrealEngine/Engine/Plugins/Tests/FunctionalTestingEditor/Source/Private/FunctionalTestingEditorModule.cpp indexing crashed.
Couldn't start rp /home/widmo/Coding/Projects/C++/RTags/rtags/bin/rp Fork failed
[100%] 8462/8469 01:38:00 ~/Coding/Projects/C++/UE4/UnrealEngine/Engine/Plugins/Tests/FunctionalTestingEditor/Source/Private/GroundTruthDataFactory.cpp indexing crashed.
Couldn't start rp /home/widmo/Coding/Projects/C++/RTags/rtags/bin/rp Fork failed
[100%] 8463/8469 01:38:00 ~/Coding/Projects/C++/UE4/UnrealEngine/Engine/Plugins/Tests/RuntimeTests/Source/RuntimeTests/Private/AutomationExpectedError.spec.cpp indexing crashed.
Couldn't start rp /home/widmo/Coding/Projects/C++/RTags/rtags/bin/rp Fork failed
[100%] 8464/8469 01:38:00 ~/Coding/Projects/C++/UE4/UnrealEngine/Engine/Plugins/Tests/RuntimeTests/Source/RuntimeTests/Private/AutomationSpec.spec.cpp indexing crashed.
Couldn't start rp /home/widmo/Coding/Projects/C++/RTags/rtags/bin/rp Fork failed
[100%] 8465/8469 01:38:00 ~/Coding/Projects/C++/UE4/UnrealEngine/Engine/Plugins/Tests/RuntimeTests/Source/RuntimeTests/Private/RuntimeTests.cpp indexing crashed.
Couldn't start rp /home/widmo/Coding/Projects/C++/RTags/rtags/bin/rp Fork failed
[100%] 8466/8469 01:38:00 ~/Coding/Projects/C++/UE4/UnrealEngine/Engine/Plugins/Tests/RuntimeTests/Source/RuntimeTests/Private/Scripting/ScriptFuzzingTests.cpp indexing crashed.
Couldn't start rp /home/widmo/Coding/Projects/C++/RTags/rtags/bin/rp Fork failed
[100%] 8467/8469 01:38:00 ~/Coding/Projects/C++/UE4/UnrealEngine/Engine/Plugins/Tests/RuntimeTests/Source/RuntimeTests/Private/Shaders/ShaderComparisonTests.cpp indexing crashed.
Couldn't start rp /home/widmo/Coding/Projects/C++/RTags/rtags/bin/rp Fork failed
[100%] 8468/8469 01:38:00 ~/Coding/Projects/C++/UE4/UnrealEngine/Engine/Plugins/Tests/RuntimeTests/Source/RuntimeTests/Private/Slate/RichTextMarkupProcessingTest.cpp indexing crashed.
Couldn't start rp /home/widmo/Coding/Projects/C++/RTags/rtags/bin/rp Fork failed
[100%] 8469/8469 01:38:00 ~/Coding/Projects/C++/UE4/UnrealEngine/Engine/Plugins/Tests/ScreenshotTools/Source/ScreenshotTools/Private/ScreenshotTools.cpp indexing crashed.
Jobs took 18.80s, (avg 0.00s). We're using 8495mb of memory.

screenshot from 2018-11-30 01-50-21

Environment (please complete the following information): Pop!_OS 18.10 (Ubuntu 18.04 Derivative) 64 bit. Clang version 7.0.0-3 (tags/RELEASE_700/final) RTags 2.21 (Commit 12499509)

sthfrnth commented 5 years ago

I encountered the same problem in an embedded project.

I guess some files with big inline raw data (e.g. unsigned char []) could trigger this problem. But I have not narrowed down to know which files.