Very slow performance when using Remote - WSL

[ahmedkrmn]

Type: LanguageService

Describe the bug

• OS and Version: Windows_NT 10.0.18363
• VS Code Version: 1.45.1
• C/C++ Extension Version: 0.28.1 I tried downgrading to previous versions but the issue still persists.
• Does this issue involve using SSH remote to run the extension on a remote machine?: Yes

Description: Code intellisense as well as auto formatting are very slow for very simple C++ files when using the Remote - WSL extension for C++ development on WSL.

uname -r output: 4.4.0-18362-Microsoft

This issue isn't there when developing on Windows.

Steps to reproduce

Create a hello world C++ file and allow formatting on save from VScode.

[Colengms]

Hi @ahmedkrmn . When remoting, the extension runs on the target machine. Are you remoting into a slow system, such as an embedded device?

Regarding the performance of formatting; The C/C++ Extension currently uses clang-format for formatting. A copy of clang-format is bundled with the extension, but a version installed on the (remote) device will be used instead if newer. You might try running clang-format directly, on the device, to see if that time is being spent by it. If so, perhaps there is a version of clang-format available that is optimized for your system. If clang-format actually completes very quickly when run directly, let us know. That might point to another issue.

[ahmedkrmn]

Hi @Colengms, The target machine is an Ubuntu WSL version 4.4.0-18362-Microsoft. I tried directly using clang-format as suggested and it formatted without any delays.

I did some further inspections and it appears that the issue is only preset when the workspace is the home directory of the WSL. When I created a new directory and used the VS remote extension to use it as the workspace, the issue disappears and formatting as well as linting is working well.

[sean-mcmanus]

I'm not reproing the issue. Is this with the "/home/" directory? It sounds like something is blocking our main input thread.

[MitchTalmadge]

I am having the same issues although my workspace is not the home directory. Autoformatting is very slow. How can I help provide info for debugging?

• OS and Version:
  • Host: Windows 10 v2004
  • WSL: Debian 9 stretch
• VS Code Version: 1.46.1
• C/C++ Extension Version: 0.28.3

[sean-mcmanus]

@MitchTalmadge Are your files under /mnt? I think /mnt access can be slow with WSL2.

If you can wait for CPU usage to drop to 0 before doing a format, that would help isolate if the formatting is slow or just being blocked by previous processing (formatting is not expected to be slow, so probably other operations are blocking the formatting).

[MitchTalmadge]

@MitchTalmadge Are your files under /mnt? I think /mnt access can be slow with WSL2.

If you can wait for CPU usage to drop to 0 before doing a format, that would help isolate if the formatting is slow or just being blocked by previous processing (formatting is not expected to be slow, so probably other operations are blocking the formatting).

They may be, I will have to double check and let you know. Thanks for the follow-up, really appreciate it

[ZijunH]

I have a similar experience on WSL2 as well. My files are stored in /mnt/ and Intellisense is extremely slow, requiring more than 3 seconds for the results to pop up. After moving the files to ~, there is no longer a delay (or a very small one). I suspect this is due to WSL2, which has suprisingly slow access speed for /mnt/

[ghost]

I have a similar experience on WSL1. There's a CPU usage surge on cpptools:

I also checked debug level log, it stayed at Formatting Engine: clangFormat for a while.

I tried to use clang-format directly from a terminal, it's fast.

The intellisnese updating and file analyzing speed is also slow.

[corneliusroemer]

Same issue here. All my files are under /home, I'm using WSL:Ubuntu-20.04. Intellisense is extremely slow - not usable. I'm using it for Python.

[pierce-martin]

Same issue for me. Files are under the home directory. Versions are:

• vscode: 1.50.1
• WSL: Ubuntu 18.04
• C/C++ extension: 1.0.1 (both local and remote)

Main issue I see is tab completion is slow and go to declaration/definition doesn't work. I see cpptools CPU usage spike as well.

[ghost]

I have a similar experience on WSL1. There's a CPU usage surge on cpptools:

I also checked debug level log, it stayed at Formatting Engine: clangFormat for a while.

I tried to use clang-format directly from a terminal, it's fast.

The intellisnese updating and file analyzing speed is also slow.

I upgraded my computer from a 2c4t VDI to a 16c16t VDI and switched from WSL1 to WSL2. The issue has been improved significantly. Though I'm not sure if it's due to CPU performance or WSL1 -> WSL2.

[stertingen]

I have the same issue when working on a ROS workspace. (ROS workspace consists of 3 directories: src, build and devel, with devel being some sort of install target location. There are many files in these directories, but not too many IMHO: 670 files in 192 directories)

VS Code 1.49.1 C/C++ extension: 1.1.2 ROS extension: 0.6.4 WSL: Ubuntu 20.04 (Linux 4.4.0-18362-Microsoft # 1049-Microsoft Thu Aug 14 12:01:00 PST 2020 x86_64 x86_64 x86_64 GNU/Linux) Host: Windows 10 Enterprise version 1909 build 18363.1198

[Kraego]

I have the same issue with haskell and C/C++, files in home dir. Haskell language server on WSL.

I think this could be a wsl or code issue, since it's happening in c/c++, python, haskell, ...

VS Code 1.52.1 WSL2: Debian Host: Windows 10 Home 1909

[klakiervx]

Same for me. cpptools sometimes hogs one full CPU core (~8% on 6-core with HT) for a couple of minutes (especially after opening another c++ file) or 10-30 seconds after modifying something in the file. IntelliSense popup often times out before it gets any data to show. Almost impossible to work like this.

VSCode 1.53.1 WSL1Ubuntu 18.04 Windows 10 Enterprise 1909

[rjkz808]

Same issue with WSL 1 and TypeScript Intellisense/type-checking when files are located in /mnt/.

VSCode 1.54.3 WSL1: Ubuntu 18.04.5 Host: Windows 10 Enterprise 1909

[mmatrosov]

I have the same issue under Linux with real Ubuntu (not WSL). But I also connect via ssh, thus it is probably related.

VSCode:

Version: 1.54.3
Commit: 2b9aebd5354a3629c3aba0a5f5df49f43d6689f8
Date: 2021-03-15T10:55:24.277Z
Electron: 11.3.0
Chrome: 87.0.4280.141
Node.js: 12.18.3
V8: 8.7.220.31-electron.0
OS: Linux x64 5.8.0-44-generic snap

Host: Ubuntu 20.04.2 LTS Remote: Ubuntu 20.04.2 LTS

"Format Document" command works instantaneously. Same for "Save without Formatting" command. But when I save with editor.formatOnSave option enabled, it might take several seconds.

[valerioformato]

Same for me. cpptools sometimes hogs one full CPU core (~8% on 6-core with HT) for a couple of minutes (especially after opening another c++ file) or 10-30 seconds after modifying something in the file. IntelliSense popup often times out before it gets any data to show. Almost impossible to work like this.

VSCode 1.53.1 WSL1Ubuntu 18.04 Windows 10 Enterprise 1909

Same for me. I have the exact same symptoms Ubuntu 20.04 running on WSL (Win10 build 19041.487) VSCode 1.54.3 C/C++ Extension Version: 1.2.2

edit: I am adding the C/C++ diagnostics log and the C/C++ extension log for one of the really long sessions. What I find worrisome is the line

Update IntelliSense time (sec): 994.465

[valerioformato]

I also tried to attach strace and then triggering one of these 100%cpu-burning moments on cpptools What i see is that the process hangs for several seconds on a call to

futex(0x7f9a2e58bb80, FUTEX_WAIT_PRIVATE, 2, NULL) = 0

The preceding calls are

read(0, "Content-Length: 217\r\n\r\n{\"jsonrpc"..., 512) = 240
read(0, 0x1d56ee0, 512)                 = ? ERESTARTSYS (To be restarted if SA_RESTART is set)
--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=4806, si_uid=1000, si_status=0, si_utime=0, si_stime=0} ---
read(4, "*", 1)                         = 1
write(7, "X\302\323\1\0\0\0\0\21\0\0\0\0\0\0\0", 16) = 16
write(5, "*", 1)                        = 1
rt_sigreturn({mask=[]})                 = 0
read(0, "Content-Length: 53\r\n\r\n", 512) = 22
read(0, "{\"jsonrpc\":\"2.0\",\"method\":\"cppto"..., 512) = 53
futex(0x7f9a2e58bb80, FUTEX_WAIT_PRIVATE, 2, NULL) = 0  ## HANGS FOR SEVERAL SECONDS

[sean-mcmanus]

@valerioformato Are you able to locate the call stack that is calling the futex? Are those the reads in your log? If so, what calls precede the reads?

[valerioformato]

Getting the stack is extremely hard, the whole WSL is completely unresponsive for several second, top freezes and gdb is stuck trying to attach to the process, and i get the feeling that by the time gdb has successfully attached the critical part is already passed.

I can try to crosscorrelate the strace with the extension log but I have to do this manually and it will take me some trial and error.

[sean-mcmanus]

Are you able to attach beforehand and then just break when the bug repros?

[valerioformato]

Unfortunately not, also the break signal doesn't get processed until the whole system unfreezes again :(

[sean-mcmanus]

There are instructions on how to get performance/profiler logging for Linux at https://github.com/microsoft/vscode-cpptools/wiki/Troubleshooting-Performance-Issues .

[valerioformato]

There are instructions on how to get performance/profiler logging for Linux at https://github.com/microsoft/vscode-cpptools/wiki/Troubleshooting-Performance-Issues .

Doesn't look like perf can run in WSL 1 :\

[sean-mcmanus]

Have you tried the Windows profiling approach? With WSL 1 the processes may still be considered Windows processes (while with WSL2 they run in some sort of Linux VM).

[valerioformato]

Have you tried the Windows profiling approach? With WSL 1 the processes may still be considered Windows processes (while with WSL2 they run in some sort of Linux VM).

Ouch, no I didn't think about that. I had been postponing the upgrade to WSL2 due to several issues with virtualization on my system. (that was also preventing me from using docker) Lately the frustration has driven me to spend the afternoon fighting with that, and I managed to finally upgrade. I'll let you know if the issue persists.

[miloxeon]

So, if I have Home Edition and thus no WSL 2.0, I'm doomed to slow performance on Windows?

[sean-mcmanus]

@mvoloskov If you search for "Windows Home edition wsl2" you should be able to find some steps to enable WSL 2.0 on Home Edition -- let us know if that doesn't work.

[sean-mcmanus]

@mvoloskov See https://docs.microsoft.com/en-us/windows/wsl/wsl2-faq -- it says "WSL 2 is available on all SKUs where WSL is currently available, including Windows 10 Home.".

[miloxeon]

Thank you all, I ended up just using Git for Windows and uninstalling WSL completely. With Alacritty, performance is stupidly fast.

[sbland]

I have the same issue but only when I also get the Workspace is too large to track changes message. The workspace isn't a git repo but contains a python venv directory that is presumably too large to track. I fixed the problem by adding **/venv/** to the watcher ignore list:

"files.watcherExclude": {
  "**/venv/**": true
}

[chriszhao001]

Same for me. But the main process which make whole PC very slow in windows task manager in "node" process. When I use visual studio code open an directory in WSL "home" directory, I can see an process "node" in task manager. after I close visual studio or move to another directory which is not in WSL "home" directory, the "node" process disappear. the node file is located in /home/xxx/.vscode-server/bin/3866c3553be8b268c8a7f8c0482c0c0177aa8bfa

Ubuntu 20.04 running on WSL (Win10 build 19042.1165) VSCode 1.59.1 C/C++ Extension Version: 1.6.0

[kilgarenone]

I got myself into Remote-Host extension from following Docker Desktop instruction to use VSCode in WSL2 environment so I can run Mongo in docker, and yes it's all so slow...any advices?

[cnkjw]

This issue seems to still exist.

• VSCode: v1.61.2
• C/C++ extension: v1.7.1
• WSL1: Ubuntu-18.04
• Win: Win10 21H1 19043.1288

[Veenkar]

How about: crtl+shift+P -> reset intellisense database?

[adams-family]

Is it possible to use VS Code locally (all extensions locally) and only edit files on the remote machine?

It's is an issue for me, as well, on all cheap VPS servers.

[michelleangela]

@adams-family

Is it possible to use VS Code locally (all extensions locally) and only edit files on the remote machine?

Short answer: In order for the language server to effectively parse a project for IntelliSense, it needs to run on the same machine where the project and compiler are on.

Example: The language server requires parsing of all files in a project in order to find and process symbols. If a project is on a remote machine, then the entire project as well as system header dependencies will need to be sent over to the local machine where the language server is running. Also, the extension queries the compiler in order to find system headers and get defines.

[adams-family]

@michelleangela Got it, makes sense! In this case, can I work with VS Code locally (on my laptop, including IntelliSense) and sync my files with the remote server on each Save?

[michelleangela]

@adams-family

sync my files with the remote server on each Save?

The C++ extension does not have a way to sync files to another machine. It only knows the context of working on the machine the extension is running on. There may be some extension or some customized way to sync files over to another machine. Or use source control to port over changes.

[adams-family]

@michelleangela Understood. I'm administering a lot of small projects where I occasionally need to connect, change a configuration file (or a source code), debug the problem and leave. The issue is when VS code server-extensions eat out the resources on the remote VPS, it becomes unresponsive. I used NetBean's remote sync for this but that project got abandoned. Thanks for the clarification anyway!

[grapereader]

Similar issue... formatting C++ files never completes on a remote host. The host has ample RAM and CPU, but everything (including vscode-server itself) is on a very slow network share.

Here's some speculation: On this system, disk reads are fast enough, but disk writes are slow. Especially with lots of small files. VSCode must not be optimized for this (is it shuffling around small intermediate files?). This would also explain poor performance with WSL1 which struggles with disk speed.

[ginkgo97]

Similar issue. I installed wsl1 on my win10 os.

• vscode version: 1.69.0
• wsl1 version: ubuntu 22.04

When I enable the Remote-WSL plugin and connect to wsl, it is slowly responsed if I use bash on wsl within vscode. But when I disconnect from wsl, and only use bash on wsl within vscode, everything is ok. I suppose this issue probably has nothing to do with cpptools, but remote wsl.

[ctjlewis]

A team member of mine switched to Mac to workaround. This issue should be prioritized based on the number of user reports.

[Colengms]

There's been a lot of traffic on this issue, which suggests many people are experiencing perf issues when using the C/C++ Extension remotely via WSL.

The C/C++ Extension runs natively on a 'remote' system. It does not operate differently specifically for WSL remoting. It's native processes are the same as used on other native Linux platforms. While it's possible that there could be something the C/C++ Extension is doing that could be less efficient on Linux than on Windows, it's likely that performance issues in WSL are specific to VS Code's WSL remoting, or WSL itself. For example, if a project is on the Windows file system and accessed through a /mnt mount point, file operations are going through an additional layer which may account for some performance impact. It's less efficient than the same workspace opened on Windows. In the past, we have seen WSL bugs specifically with /mnt mount points.

For many WSL performance issues, the resolution would be unlikely to come from the C/C++ Extension. Consider investigating whether similar performance issues can be reproduced with VS Code itself or other extensions, and open issues in the VS Code repo, if so.

[DarrenLevine]

I found a workaround for this issue:

1. Install the jkillian.custom-local-formatters extension
2. Add the following lines to your /.vscode/settings.json file:

   "customLocalFormatters.formatters": [
   {
       "command": "clang-format",
       "languages": [
            "cpp"
       ]
   }
   ]

3. Open any C++ file and run "> Format Document" in VSCode's command palette.
4. Vscode will ask you which formatter to use, select the custom-local-formatters option as the new default.
5. From then on, when you save a C++ document, it will ignore the slower C++ extension's default formatter, and use clang-format directly, which is much faster.

Note that you can also use a ".clang-format" file with this method, and set the BasedOnStyle option in that file to LLVM to get a format similar to what the vscode C++ extension's formatter uses as a default configuration.

[sean-mcmanus]

As I believe I mentioned earlier, this is caused by our formatting operating getting stuck behind an IntelliSense update. It's possible this could be fixed or improved by https://github.com/microsoft/vscode-cpptools/issues/7759 .