search

microsoft / vscode

Visual Studio Code
https://code.visualstudio.com
MIT License
162.42k stars 28.62k forks source link

vscode freezes every 3-5minutes randomly #124723

Closed alexbi29 closed 3 years ago

alexbi29 commented 3 years ago

Version: 1.56.2 Commit: 054a9295330880ed74ceaedda236253b4f39a335 Date: 2021-05-12T17:44:30.902Z (2 wks ago) Electron: 12.0.4 Chrome: 89.0.4389.114 Node.js: 14.16.0 V8: 8.9.255.24-electron.0 OS: Darwin x64 20.5.0

remote ssh to linux host.

only extensions installed: remote-ssh, c/c++

Log (Window) shows this:

[2021-05-26 19:46:58.540] [renderer4] [warning] {}
[2021-05-26 19:47:03.024] [renderer4] [warning] {}
[2021-05-26 19:48:00.725] [renderer4] [error] Maximum call stack size exceeded: RangeError: Maximum call stack size exceeded
    at l._entries (file:///Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:62:7277)
    at l._entries (file:///Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:62:7296)
    at _entries.next (<anonymous>)
    at l._entries (file:///Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:62:7383)
    at _entries.next (<anonymous>)
...
vscodebot[bot] commented 3 years ago

(Experimental duplicate detection) Thanks for submitting this issue. Please also check if it is already covered by an existing one, like:

dbaeumer commented 3 years ago

@alexdima @jrieken do we still have means to map locations in minified code to un-minified code?

jrieken commented 3 years ago

Pretty sure it's this: _entries but I don't know yet how/why

alexbi29 commented 3 years ago

let me know if i can help debug this. It also seem to happens across platforms - we have some vscode on windows and this issue is there too. Uninstalled all plugins except remote-ssh and it's still happening.

jrieken commented 3 years ago

@alexbi29 Can you provide the whole call-stack? Like, I wonder if ... cut off the actual start of the call-stack

alexbi29 commented 3 years ago

@jrieken don't think so (see below). Symptoms are: it freezes for like 10-15sec then unfreezes for 2 seconds and then another freeze for 2 seconds and after that it's useable again. Removed all plugins (except remote-ssh) and it is still happening.

[2021-05-28 00:20:55.960] [renderer1] [error] Maximum call stack size exceeded: RangeError: Maximum call stack size exceeded at l._entries (file:///Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:62:7277) at _entries.next () ....

jrieken commented 3 years ago

Can you try this with VS Code Insiders? I am planning to add some more logging to understand the root case and Insiders would be an easy way for you to report this

alexbi29 commented 3 years ago

Still getting the same freezes w/ insiders (w/ just remote-ssh plugin enabled editing cpp code). Also, if it's helpful - really started to happen about 2-3 weeks ago.

[2021-05-28 17:09:10.464] [renderer1] [error] Maximum call stack size exceeded: RangeError: Maximum call stack size exceeded at r._entries (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:62:7277) at r._entries (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:62:7296) at _entries.next (<anonymous>) at r._entries (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:62:7383) at _entries.next (<anonymous>) at r._entries (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:62:7383) at _entries.next (<anonymous>) at r._entries (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:62:7383) at _entries.next (<anonymous>) at r._entries (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:62:7383) at _entries.next (<anonymous>) at r._entries (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:62:7383) at _entries.next (<anonymous>) at r._entries (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:62:7383) at _entries.next (<anonymous>) at r._entries (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:62:7383) at _entries.next (<anonymous>) at r._entries (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:62:7383) at _entries.next (<anonymous>) at r._entries (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:62:7383) at _entries.next (<anonymous>) at r._entries (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:62:7383) at _entries.next (<anonymous>) at r._entries (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:62:7383) at _entries.next (<anonymous>) at r._entries (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:62:7383) at _entries.next (<anonymous>) at r._entries (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:62:7383) at _entries.next (<anonymous>) at r._entries (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:62:7383) at _entries.next (<anonymous>) at r._entries (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:62:7383) at _entries.next (<anonymous>) at r._entries (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:62:7383) at _entries.next (<anonymous>) at r._entries (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:62:7383) at _entries.next (<anonymous>) at r._entries (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:62:7383) at _entries.next (<anonymous>) at r._entries (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:62:7383) at _entries.next (<anonymous>) at r._entries (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:62:7383) at _entries.next (<anonymous>) at r._entries (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:62:7383) at _entries.next (<anonymous>) at r._entries (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:62:7383) at _entries.next (<anonymous>) at r._entries (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:62:7383) at _entries.next (<anonymous>) at r._entries (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:62:7383) at _entries.next (<anonymous>) at r._entries (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:62:7383) at _entries.next (<anonymous>) at r._entries (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:62:7383) at _entries.next (<anonymous>) at r._entries (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:62:7383) at _entries.next (<anonymous>) at r._entries (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:62:7383) at _entries.next (<anonymous>) at r._entries (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:62:7383) at _entries.next (<anonymous>) at r._entries (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:62:7383) at _entries.next (<anonymous>) at r._entries (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:62:7383) at _entries.next (<anonymous>) at r._entries (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:62:7383) at _entries.next (<anonymous>) at r._entries (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:62:7383) at _entries.next (<anonymous>) at r._entries (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:62:7383) at _entries.next (<anonymous>) at r._entries (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:62:7383) at _entries.next (<anonymous>) at r._entries (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:62:7383) at _entries.next (<anonymous>) at r._entries (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:62:7383) at _entries.next (<anonymous>) at r._entries (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:62:7383) at _entries.next (<anonymous>) at r._entries (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:62:7383) at _entries.next (<anonymous>) at r._entries (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:62:7383) at _entries.next (<anonymous>) at r._entries (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:62:7383) at _entries.next (<anonymous>) at r._entries (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:62:7383) at _entries.next (<anonymous>) at r._entries (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:62:7383) at _entries.next (<anonymous>) at r._entries (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:62:7383) at _entries.next (<anonymous>) at r._entries (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:62:7383) at _entries.next (<anonymous>) at r._entries (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:62:7383) at _entries.next (<anonymous>) at r._entries (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:62:7383) at _entries.next (<anonymous>) at r._entries (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:62:7383) at _entries.next (<anonymous>) at r._entries (file:///Applications/Visual Studio Code - Insiders.app/Contents/Resources/app/out/vs/workbench/workbench.desktop.main.js:62:7383)

jrieken commented 3 years ago

Ok, thanks for checking. I have pushed some logging an tomorrows (June 1st) insiders should be printing a message/trace like this potential CYCLE detected. Please check the console for that - it likely won't freeze anymore

worstpractice commented 3 years ago

@jrieken I'm having the same issue currently, on Windows 10.

I'm on latest stable of everything (AFAIK) on this machine.

jrieken commented 3 years ago

ping to check the console for the potential CYCLE detected message and its trace

alexbi29 commented 3 years ago

Freeze just happened again, but log is different this time. May not be related: [2021-06-02 14:17:59.643] [renderer1] [warning] {} [2021-06-02 14:18:50.556] [renderer1] [info] [remote-connection][ExtensionHost][f166f…][reconnect] starting reconnecting loop. You can get more information with the trace log level. [2021-06-02 14:18:50.556] [renderer1] [info] [remote-connection][ExtensionHost][f166f…][reconnect] resolving connection... [2021-06-02 14:18:50.556] [renderer1] [info] [remote-connection][ExtensionHost][f166f…][reconnect] connecting to 127.0.0.1:58774... [2021-06-02 14:18:50.739] [renderer1] [info] [remote-connection][ExtensionHost][f166f…][reconnect] reconnected!

Also see bunch of errors in "Log (Remote Extension Host)": [2021-06-02 13:17:09.131] [exthost] [error] [ms-vscode.cpptools] provider FAILED [2021-06-02 13:17:09.132] [exthost] [error] Canceled: Canceled at FoldingRangeProvider.<anonymous> (/home/user/.vscode-server-insiders/extensions/ms-vscode.cpptools-1.4.0/dist/main.js:35700:23) at Generator.next (<anonymous>) at fulfilled (/home/user/.vscode-server-insiders/extensions/ms-vscode.cpptools-1.4.0/dist/main.js:35674:58) at runMicrotasks (<anonymous>) at runNextTicks (internal/process/task_queues.js:58:5) at processImmediate (internal/timers.js:434:9) [2021-06-02 13:19:09.061] [exthost] [error] [ms-vscode.cpptools] provider FAILED [2021-06-02 13:19:09.061] [exthost] [error] Canceled: Canceled at FoldingRangeProvider.<anonymous> (/home/user/.vscode-server-insiders/extensions/ms-vscode.cpptools-1.4.0/dist/main.js:35700:23) at Generator.next (<anonymous>) at fulfilled (/home/user/.vscode-server-insiders/extensions/ms-vscode.cpptools-1.4.0/dist/main.js:35674:58)

alexbi29 commented 3 years ago

It's happening again. Uninstalled all plugins (went to remote server and deleted .vscode-server/caches). Don't see any logs in Log (Window) anymore. If "unresponsive window" pops up then i see this in "Log (Main)": [2021-06-02 15:16:42.577] [main] [error] CodeWindow: detected unresponsive

Freezes are still 15-20 seconds, unfreeze for 2-3 sec and then another freeze for 2-3.

One thing that stands out - it only happens right after i do a build (through tasks.json that really just calls bazel build) and click on one of the syntax errors from gcc to go to file/line.

Here's task that builds: { "label": "build all", "type": "shell", "command": "bazel build ...", "group": { "kind": "build", "isDefault": true } }

jrieken commented 3 years ago

One thing that stands out - it only happens right after i do a build (through tasks.json that really just calls bazel build)

Oh, that good news. If it reproduces consistently, than please try to profile the renderer process. E.g start profiling, do whatever causes the freeze, stop profiling

Hit F1 > Toggle Developer Tools. In the overflow menu select 'More Tools > JavaScript Profiler'. In there select start, let it profile for 30 to 60 seconds, stop it, save it to a file and attach the file here. Thanks.

https://user-images.githubusercontent.com/1794099/120606621-a97e0180-c44f-11eb-8e32-9fdafdc949ec.mov

alexbi29 commented 3 years ago

@jrieken here it is. i was scrolling up and down when freeze happened. CPU-20210603T144656.cpuprofile.zip

jrieken commented 3 years ago

@alexbi29 with what version of VS Code is that happening? Are you still on insiders?

jrieken commented 3 years ago

After looking at the CPU profile, I am making a qualified guess and I say this is due to a file watching, e.g receiving a very, very large file event. The first plateau is receiving via IPC and the second plateau is FileChangesEvent > getAdded() > getOfType() > TenarySearchTree > iterator > _entries. The seems to eagerly retrieve all items from the change event and assuming the event is huge that will take a long time.

the two plateaus, note that hight represent call stack depth, not CPU load

Screen Shot 2021-06-04 at 12 45 20

_calls to _entries_

Screen Shot 2021-06-04 at 12 45 41
jrieken commented 3 years ago

I can work on a change to make _entries not use recursion anymore (to prevent the stack overflow) but it seems we shouldn't get so far...

Also adding @isidorn because FileChangesEvent#getAdded and friends are deprecated.

jrieken commented 3 years ago

I can work on a change to make _entries not use recursion anymore (to prevent the stack overflow) but it seems we shouldn't get so far...

@alexbi29 Just to double & triple check that that you use insiders. That has a breaker for an endless loop (in case the underlying tree structure is bad) but it actually seems to caused by pure overload...

isidorn commented 3 years ago

Tried to get out of using getAdded in the explorer but I think I could not fully do it (e.g not using it caused a performance problem in the explorer). More details here https://github.com/microsoft/vscode/issues/108165 I can also refresh my memory...

bpasero commented 3 years ago

I see 2 solutions:

I think we can optimize the ternary search tree but I also think that it is not the best data structure for the deprecated methods.

Thoughts?

(not sure if this is May or June, looks like a workaround would be to configure files.watcherExclude setting, @alexbi29 you can try to exclude the build folders that generate many files when you run the task to be unblocked)

jrieken commented 3 years ago

Thoughts?

We can optimize the TST further or move off the deprecated functions but the majority of the time is actually not spend there. The first plateau is much longer and based on my understanding that just receiving data. So, I believe we should be pragmatic and the watcher process should simply not shuffle so much data into the renderer. Ideally, we can reproduce or debug or log to know how many events are send but I'd say at a certain limit we should just drop further events

bpasero commented 3 years ago

Yeah, the file watcher is a potential source of unlimited events as there is no debouncing going on and we never know how many file events are generated. At least this is running in a child process for that reason to take work off the renderer but it is true that data is still just sent into the renderer without bounds.

We can add some debouncing and throttling I guess, though finding the right number that does not cause the UI to react slow on file events is the challenge I guess. Not sure if the extension host has similar IPC debouncing @alexdima ?

alexdima commented 3 years ago

Not sure if the extension host has similar IPC debouncing @alexdima ?

No debouncing for throttling reasons implemented.

alexbi29 commented 3 years ago

@jrieken what i used to repro the issue: Version: 1.57.0-insider (Universal) Commit: a62df5d6c53e844bfdc50694f943c3f34b6ff50e Date: 2021-06-03T17:36:36.265Z (22 hrs ago) Electron: 12.0.9 Chrome: 89.0.4389.128 Node.js: 14.16.0 V8: 8.9.255.25-electron.0 OS: Darwin x64 20.5.0

it's reproducible every time i compile the code (run that task).

jrieken commented 3 years ago

@alexbi29 Thanks for clarifying

it's reproducible every time i compile the code (run that task).

This would be a good change to count and log the number of file events during a period of time

alexbi29 commented 3 years ago

@bpasero I do have exclude set up, so it should not be parsing those. I suspect there's some issue in vscode related to the way bazel works w/ symlinks. I see some other inconsistencies in code browsing: when clicking on "go to definition" files are opened from ~/.cache/bazel/... instead of workspace folders (but directly opening a file w/ command-p opens it in a second window from the workspace location). Workspace has bazel- name where it stores all deps (those are simlinks in bazel) and I use multi-root workspace to link to it. E.g.: "folders": [ { "name": "nts-engine", "path": "../" }, { "name": "deps", "path": "../bazel-engine/external/deps/source" } ], "settings": { "files.exclude": { "**/.git": true, "**/__pycache__": true, "**/bazel-*": true, "vendor/": true, "**/bin/": true }, ... }

bpasero commented 3 years ago

@alexbi29 if you run VSCode with --verbose, we log every file watcher file event that we encounter with the full path. This would be a way to figure out if the exclude pattern uses a different path than the ones we actually encounter, can you try?

Please see if you can find any logged output that could help us find the source of the issue:

Thanks!

jrieken commented 3 years ago

Tried to get out of using getAdded in the explorer but I think I could not fully do it (e.g not using it caused a performance problem in the explorer). More details here #108165 I can also refresh my memory...

Looking at how you use getAdded: Would it make sense to retrieve the changes one by one through an iterator and not as wholesome array? It seems that you only set a boolean (shouldRefresh) and that you don't actually need all changed/added files at once. See that the expensive part here is creating the array (which uses an underlying iterator). So, instead of N this could run on average in N/2

jrieken commented 3 years ago

fyi @bpasero it seems that getUpdated is unused and ready for removal

bpasero commented 3 years ago

@jrieken done, I think in my usage here:

https://github.com/microsoft/vscode/blob/903e219ccb8f8a0bccf8ffce33b79cb66bd9a4f4/src/vs/workbench/contrib/files/browser/editors/textFileEditor.ts#L75-L78

I could also use a iterator approach if that helps.

jrieken commented 3 years ago

I could also use a iterator approach if that helps.

I don't see an "early return" condition as in the explorer so I don't think so. Tho, maybe the text editor view state knows what it is looking for and than things can be turned around

bpasero commented 3 years ago

Yeah there is no early return but in the end we are just looping over the entries here:

https://github.com/microsoft/vscode/blob/b82b90c1000a953a2caf6f5452326f202518ec49/src/vs/workbench/browser/parts/editor/textEditor.ts#L287-L291

So there is no benefit from putting all the changes into an array upfront.

jrieken commented 3 years ago

It only allows to change the API to return an iterable but it doesn't help with this performance issue

bpasero commented 3 years ago

Yeah I would still add some debouncing on the event creation side in any case.

jrieken commented 3 years ago

I have pushed a change to avoid recursion when iterating over the TST. This should prevent the stack overflow due to too much data. The default call stack size is 11384 which means we are facing path more segments than that...

bpasero commented 3 years ago

@alexbi29 is your project open source, is there a way for me to replay your steps to see it hang?

bpasero commented 3 years ago

Pushed a change to iterate over the ternary search tree and avoid having to create an array.

@isidorn please review https://github.com/microsoft/vscode/commit/a0a9141869093495444eb4c107da6f5c173d456d#diff-20f01ca14a8b2d9406b1c703845f3d86d3909a8451226b32bf2f9d382f196ce0

isidorn commented 3 years ago

@bpasero the change looks good, thanks for doing it 👍

alexbi29 commented 3 years ago

Sorry for the delay. Enabled --verbose on the most recent insiders. I see tons of events that add tons (~100-500k) of files after each compilation (even if no changes are made): ` ..........

[2021-06-11 09:48:43.607] [remoteagent] [trace] [File Watcher (nsfw)] >> normalized [ADDED] /home/user/src/vertex/bazel-vertex/.vscode [2021-06-11 09:48:43.607] [remoteagent] [trace] [File Watcher (nsfw)] >> normalized [ADDED] /home/user/src/vertex/bazel-vertex/alpha [2021-06-11 09:48:43.607] [remoteagent] [trace] [File Watcher (nsfw)] >> normalized [ADDED] /home/user/src/vertex/bazel-vertex/bin [2021-06-11 09:48:43.607] [remoteagent] [trace] [File Watcher (nsfw)] >> normalized [ADDED] /home/user/src/vertex/bazel-vertex/blogAnalyzer [2021-06-11 09:48:43.607] [remoteagent] [trace] [File Watcher (nsfw)] >> normalized [ADDED] /home/user/src/vertex/bazel-vertex/cdef .... [2021-06-11 10:05:59.117] [remoteagent] [trace] [File Watcher (nsfw)] >> normalized [ADDED] /home/user/src/vertex/bazel-vertex/external/vendor/packages [2021-06-11 10:05:59.117] [remoteagent] [trace] [File Watcher (nsfw)] >> normalized [ADDED] /home/user/src/vertex/bazel-vertex/external/vendor/version [2021-06-11 10:05:59.117] [remoteagent] [trace] [File Watcher (nsfw)] >> normalized [CHANGED] /home/user/src/vertex/bazel-out/_tmp/actions [2021-06-11 10:05:59.117] [remoteagent] [trace] [File Watcher (nsfw)] >> normalized [CHANGED] /home/user/src/vertex/bazel-out/volatile-status.txt [2021-06-11 10:05:59.117] [remoteagent] [trace] [File Watcher (nsfw)] >> normalized [CHANGED] /home/user/src/vertex/bazel-out/stable-status.txt

`

bazel-vertex is created by bazel (https://bazel.build/), looks like it changes somehow every time i run compilation (even if it does not lead to any changes).

Structurally bazel-vertex is a symlink to bazel cache, but I don't see symlink change across compilations: /home/user/src/vertex/bazel-vertex -> /home/user/.cache/bazel/_bazel_user/785163ba30938911fdd6b18addc8a2e5/execroot/main

It's likely that bazel first cleans up everything in current dir (and deletes bazel-vertex simlink) and then re-creates it. Even though underlying structure does not change vscode removes and then re-adds all files there).

Other thing is i have exclusions set up (and it works as these files don't show up in the explorer): e.g. "files.exclude": { ..., "*/bazel-": true, ... }

bpasero commented 3 years ago

@alexbi29 is the performance any better with todays insider build where some fixes already went in?

You can give our preview releases a try from: https://code.visualstudio.com/insiders/

alexbi29 commented 3 years ago

Nope, still locks up the same way, updated to most recent insiders version 15 minutes ago.

I wonder if there's simple way to disable file watcher completely? Can press refresh button if project tree needs to be updated.

bpasero commented 3 years ago

You need to configure the excludes via files.watcherExclude. I think our rationale initially was that files.exclude is only for the explorer and should not tamper with the watcher because a lot of things on top of the watcher might rely on file events being emitted properly.

In your case, if you configure files.watcherExclude with those patterns it should probably be fine.

I will take a look at reproducing the hang with your instructions. Do you have a boilerplate for me to get to the build step to reproduce?

alexbi29 commented 3 years ago

hmhm, you might be able to get away w/ simpler repro. Just mv some large folder to project dir and watcher should pick it up and lock vscode? (+ add exclude for explorer to make sure it's not affecting ui?)

update: tried files.watcherExclude and UI still locks up after complile.

bpasero commented 3 years ago

@alexbi29 having a real world sample like yours would be nice, can you provide this info or is it a closed source project?

bpasero commented 3 years ago

@alexbi29 actually meanwhile I was able to setup bazel and I am compiling the examples from bazel-vertex and I see a couple of thousand file events emitted. That is not a lot though and does not result in any slowness in my system. Maybe you do something different.

Since you once created a CPU profile with the older version of VSCode, I am curious if you could do another trace based on this version of VSCode to compare it: https://code.visualstudio.com/insiders/

bpasero commented 3 years ago

I cannot really wrap my head around what best to do here...

Throttle on the watcher side

Throttle in the renderer

I tend towards just implementing the throttling in the renderer. But imho the only real solution and way out here is to change the watcher fundamentally away from a push model to a model where clients register the paths they are interested in and then we would only send those changes over the wire and would only build ternary search trees for those. Clients would not be able to register a watcher for *.* but would have to be specific, maybe at least indicate the file extension and root folder they are interested in.

bpasero commented 3 years ago

I pushed a change to throttle file events with some heuristics:

A warning is printed to the log in case the buffer is full and a trace message is printed if the watcher is busy catching up. I originally thought about making this more user visible, e.g. via progress message in the status bar, but I think that would be overkill.

Opened https://github.com/microsoft/vscode/issues/126809 for a overall better API for file watching in general.

alexbi29 commented 3 years ago

sorry for being MIA. It's not open source project, unfortunately. i still see slowdowns every time bazel compiles the code. Things to note: in my case bazel- folder contains a lot of files (500k++) - bazel pulls all project dependencies this way. Even though nothing is changed, deletion and re-creation of that symlink make vscode think it's new dir and causes it to rescan it. One solution could be to traverse symlinks and use these as keys to avoid sending the same data over and over again if symlink is updated.

bpasero commented 3 years ago

@alexbi29 oh wow, this is good information. To clarify, the file system you are connected to is a SSH linux machine right? I wonder if you are seeing https://github.com/microsoft/vscode/issues/3998 possibly, which is another file watcher related performance issue.