bug: Excessive memory usage

[tjzel]

Summary

Some time ago I enabled clangd in react-native-reanimated:

• https://github.com/software-mansion/react-native-reanimated/pull/6430

I don't know how big is this project compared to the others which use clangd. We compile files that are application-specific, library-specific and platform-specific.

Recently when I had compile-commands.json for our iOS target my Mac started lagging and in the Activity Monitor I saw clangd using 80GBs of memory and 20 threads. When I looked into VSCode to see claimed memory usage in the CLANGD MEMORY USAGE pane it reported only around 50MBs.

Unfortunately I haven't taken any log dumps back then so I cannot provide them. I think the memory issue was also correlated with my compile-commands.json being degenerated after some XCode build issues.

Are such issues known to the team? I understand that it might be the fault of clangd, not the extension, but the extension failed to report the actual memory usage.

Once the issue reappears I'll make sure to gather relevant logs and update this issue.

System information

Clangd version: Homebrew clangd version 18.1.8 Features: mac+xpc Platform: arm64-apple-darwin23.6.0

Clangd extension version: v0.1.29

System: macOS 14.6.1

[HighCommander4]

There's a known OOM that was fixed in clangd 19 (https://github.com/clangd/clangd/issues/1384), affecting specifically large .h files with many preprocessor branches (typically encountered in single-header libraries like miniaudio.h), and another OOM (not specific to large header files) fixed recently on trunk (https://github.com/llvm/llvm-project/issues/70930).

Hard to say though if you're hitting either of these or something else.

[tjzel]

Is there a way to opt-in for clangd 19? I don't see it available on Homebrew or on GitHub. Would I need to compile it from source?

[HighCommander4]

Is there a way to opt-in for clangd 19? I don't see it available on Homebrew or on GitHub. Would I need to compile it from source?

Binary packages should appear at https://github.com/llvm/llvm-project/releases/tag/llvmorg-19.1.0 within a few days.

[tjzel]

Hey @HighCommander4 I ran into the issue again. I collected the logs.

The memory usage spike happened around 15:00. At that time I was changing and deleting numerous files, both for Android and iOS codebases (some of the codebase is shared, but not all of it). Many errors were present.

On Android everything was smooth, problems started when I compiled iOS code with XCode. I'm using beta XCode version now but it happened on stable previously.

Our pipeline for obtaining compile-commands.json differs for Android and XCode. I haven't found a compilation flag which would generate the whole file. Instead I obtain fragments of it with -gen-cdb-fragment-path (reference). Later on I merge all the fragments into one JSON. The script isn't smart, but it's honest work 🧑‍🌾.

The issue seems to arise when after a successful compilation in XCode there's a failed one. Perhaps then then some .json framents are degenerated but I haven't investigated that.

I don't know how much I can trust the Activity Monitor but this doesn't seem like a memory leak. Used memory oscillated between 10 - 40 GBs which seemed to have some reflection in the server logs, memory usage dropped significantly then steadily rose again once some rebuild of some index was claimed.

When I'm finish writing this comment, the clangd server memory usage dropped to 10-80MBs.

Here are the logs and some screenshots:

10-clangd.log

[HighCommander4]

Well I guess it's clear that you're not running into either of the OOM bugs mentioned earlier, since those have different symptoms (the memory usage keeps growing).

It's also clear that there is at the least a bug in the memory usage report clangd sends to the client, since a significant amount of memory is being used somewhere but not reported.

But 10-40 GB of memory usage seems excessive as you say, especially as you're not using --pch-storage=memory, so that's likely to itself be a bug somewhere.

I wonder if it's perhaps the clang parser exhibiting excessive (but not quite unlimited) memory usage during the building of a single preamble (or a subset of individual preambles), which is then flushed to disk resulting in the memory usage dropping off. Can you try running clangd with TMPDIR=/some/dedicated/directory in the environment, and see whether you observe disk usage in that directory during or after the excessive memory usage to be on the order of (or exceeding) 10-40 GB?