Open woody77 opened 2 years ago
jonas-schievink
commented
2 years ago Hmm, the only place where I know we're allocating semi-persistent data is the "link shortener" added here: https://github.com/rust-lang/rust-analyzer/pull/12277/files
Does removing that code fix the issue?
woody77
commented
2 years ago Something that I've noticed, which I think is a vscode issue (not RA), is that if you launch vscode itself with --max-memory=8192, while --max-old-space-size=8192 is passed to all of the locally-running node processes, it doesn't get passed to the remote server (verified in the vscode Help -> Process Explorer, where hovering over a process shows you the whole command line for it).
And then if you DO go off and edit (remote) <commit>/bin/code-server to pass --max-old-space-size to node when starting, it doesn't pass it to the (remote) extension host (but only the extension host):
\_ sh ~/.vscode-server/bin/<commit>/bin/code-server --start-server...
\_ ~/.vscode-server/bin/<commit>/node --max-old-space-size=8192 ~/.vscode-server/bin/<commit>/out/server-main.js --start-server --host=127.0.0.1 ...
\_ ~/.vscode-server/bin/<commit>/node --max-old-space-size=8192 ~/.vscode-server/bin/<commit>/out/bootstrap-fork --type=ptyHost
| \_ /bin/zsh
\_ ~/.vscode-server/bin/<commit>/node /usr/local/google/home/aaronwood/.vscode-server/bin/<commit>/out/bootstrap-fork --type=extensionHost --transformURIs --useHostProxy=false
\_ ~/.vscode-server/bin/<commit>/node --max-old-space-size=8192 ~/.vscode-server/bin/<commit>/out/bootstrap-fork --type=fileWatcherHmm, the only place where I know we're allocating semi-persistent data is the "link shortener" added here: https://github.com/rust-lang/rust-analyzer/pull/12277/files
Does removing that code fix the issue?
I can try that locally (probably won't get to it for a few days).
However, I can confirm that this is prompted by changing files. Even files that aren't rust files (like our BUILD.gn files). And, we have a very large number of files in our source dir, (and without telling vscode otherwise, many more in our out/ dir).
woody77
commented
2 years ago Further, from running locally with code --max-memory=8192, I haven't seen the issue, nor do I see whatever the usage of memory is, which makes me think that it's a very short-lived allocation of a lot of heap data. And whatever it is, it's changed recently (last few weeks).
woody77
commented
2 years ago Further, from running locally with
code --max-memory=8192, I haven't seen the issue, nor do I see whatever the usage of memory is, which makes me think that it's a very short-lived allocation of a lot of heap data. And whatever it is, it's changed recently (last few weeks).
And that continues to be the case. I'm using:
"files.maxMemoryForLargeFilesMB": 8192in my settings.json, which seems to have resolved the issue for LOCAL operation only (not with vscode-remote). It appears that how that config setting is handled with the remote server is different, and so it hasn't fully resolved the issue for me.
I can 100% reproduce the issue within seconds of opening a Rust file, I just need edit some other file. In which case it looks like it's some very-short-lived-but-very-large memory usage. In digging around, it seems that RegEx match results are a js closure, and therefore aren't GC'd in the same way, because they're compiled (and the prevalence of RegEx in the above stack traces is suspicious). Perhaps something using regex's on file paths in the extension as change notifications arrive?
woody77
commented
2 years ago More investigation has led me to think that this is due to the extensionHost and file-watching.
I can consistently control whether or not I see the issue through use of the rust-analyzer.files.watcher configuration:
client -> crashes very consistently
notify -> doesn't crash
After hacking up the extension js to log heap space stats, I see that when using client (the extension host itself?) for file watching notifications, that the code_space heap can grow by >30MB when I touch a single file (even non-rust file). I have ~2.7M files in my project dir (including my multiple cross-compilation outdirs).
This looks very much like it could be similar to: https://bugs.chromium.org/p/v8/issues/detail?id=10441
Which would make this an issue in the VSCode extension host, not in RA, but is provoked by RA registering for DidFileChange notifications.
woody77
commented
2 years ago I've opened an issue with VSCode: https://github.com/microsoft/vscode/issues/153154
We'll see what happens there.
rust-analyzer version: rust-analyzer version: 0.0.0 (519d7484f 2022-06-15)
vscode version info: Version: 1.68.1 Commit: 30d9c6cd9483b2cc586687151bcbcd635f373630 Date: 2022-06-14T12:52:13.188Z (3 days ago) Electron: 17.4.7 Chromium: 98.0.4758.141 Node.js: 16.13.0 V8: 9.8.177.13-electron.0 OS: Darwin x64 21.5.0
A number of our developers (including myself) are seeing frequent crashes of the VSCode Extension Host. I've narrowed it down to RA, using extension bisect. I haven't been able to reproduce with a smaller codebase, however.
The issue usually occurs while editing a Rust source file, but I have had a few reports of developers just opening VSCode, and watching the extension host crash immediately in a loop (with Rust source files open, to trigger loading of the extension).
Stack traces from the extension host are not terribly helpful, as it's OOM'ing, and the stack is different for each occurence:
rust-analyzer status (which gives a sense of the size of our workspace):
The language server itself is using about 2GB of memory, and when I have DevTools or the VSCode node.js debugger attached, I see about 450MB of heap usage.
If I run the bundled
nodedirectly, and runv8.getHeapStatistics(), it returns:which does not line up with the levels seen above (which imply either a much smaller heap limit, or an attempt to allocate something too large to fit in the heap at all.