Language server crash with version 1.14.x with a recursive environment variable on Windows

[psclkhoury]

Environment

• OS and Version: Windows 10
• VS Code Version: 1.76.2
• C/C++ Extension Version: 1.14.4

Bug Summary and Steps to Reproduce

I open a folder which contains only a c_cpp_properties.json. After a bit of time I get a notification that the C/C++ language server has crashed and I see this in the output:

[Error - 2:27:00 PM] Sending request cpptools/queryCompilerDefaults failed.
  Message: Pending response rejected since connection got disposed
  Code: -32097 
[Error - 2:27:00 PM] The language server crashed. Restarting...

[Error - 2:27:00 PM] Sending document notification textDocument/didOpen failed
Error: Unsupported client
    at DefaultClient.<anonymous> (c:\Users\my-username\.vscode\extensions\ms-vscode.cpptools-1.14.4-win32-x64\dist\main.js:41758:23)
    at Generator.next (<anonymous>)
    at c:\Users\my-username\.vscode\extensions\ms-vscode.cpptools-1.14.4-win32-x64\dist\main.js:40304:71
    at new Promise (<anonymous>)
    at __webpack_modules__.9325.__awaiter (c:\Users\my-username\.vscode\extensions\ms-vscode.cpptools-1.14.4-win32-x64\dist\main.js:40300:12)
    at DefaultClient.queueTask (c:\Users\my-username\.vscode\extensions\ms-vscode.cpptools-1.14.4-win32-x64\dist\main.js:41735:16)
    at DefaultClient.requestWhenReady (c:\Users\my-username\.vscode\extensions\ms-vscode.cpptools-1.14.4-win32-x64\dist\main.js:41793:21)
    at c:\Users\my-username\.vscode\extensions\ms-vscode.cpptools-1.14.4-win32-x64\dist\main.js:47697:56
    at Generator.next (<anonymous>)
    at c:\Users\my-username\.vscode\extensions\ms-vscode.cpptools-1.14.4-win32-x64\dist\main.js:47681:71
    at new Promise (<anonymous>)
    at __webpack_modules__.9696.__awaiter (c:\Users\my-username\.vscode\extensions\ms-vscode.cpptools-1.14.4-win32-x64\dist\main.js:47677:12)
    at Object.didOpen (c:\Users\my-username\.vscode\extensions\ms-vscode.cpptools-1.14.4-win32-x64\dist\main.js:47694:45)
    at c:\Users\my-username\.vscode\extensions\ms-vscode.cpptools-1.14.4-win32-x64\dist\main.js:68549:50
    at Array.forEach (<anonymous>)
    at DidOpenTextDocumentFeature.register (c:\Users\my-username\.vscode\extensions\ms-vscode.cpptools-1.14.4-win32-x64\dist\main.js:68539:42)
    at DidOpenTextDocumentFeature.initialize (c:\Users\my-username\.vscode\extensions\ms-vscode.cpptools-1.14.4-win32-x64\dist\main.js:68527:18)
    at LanguageClient.initializeFeatures (c:\Users\my-username\.vscode\extensions\ms-vscode.cpptools-1.14.4-win32-x64\dist\main.js:61402:21)
    at LanguageClient.doInitialize (c:\Users\my-username\.vscode\extensions\ms-vscode.cpptools-1.14.4-win32-x64\dist\main.js:60915:18)
    at async LanguageClient.start (c:\Users\my-username\.vscode\extensions\ms-vscode.cpptools-1.14.4-win32-x64\dist\main.js:60797:13)

I only have a c_cpp_properties.json and no compile_commands.json. The c_cpp_properties.json file has the following entries:

{
  "configurations": [
    {
      "name": "Win32",
      "includePath": [
        "${workspaceFolder}/**",
        ...
      ],
      "defines": [
        ...
      ]
    }
  ],
  "version": 4
}

If I downgrade to version 1.13.9 I don't have that problem.

Configuration and Logs

The Log Diagnostics command is not working on version `1.14.x`, but here is the (reduced because of company policies) one from version `1.13.9`:

-------- Diagnostics - 3/20/2023, 2:37:07 PM
Version: 1.13.9
Current Configuration:
{
    "name": "Win32",
    "includePath": [
        "${workspaceFolder}/**",
        ...
    ],
    "defines": [
        ...
    ],
    "compilerPathIsExplicit": false,
    "cStandardIsExplicit": true,
    "cppStandardIsExplicit": true,
    "intelliSenseModeIsExplicit": true,
    "macFrameworkPath": [],
    "windowsSdkVersion": "10.0.19041.0",
    "cStandard": "c17",
    "cppStandard": "c++20",
    "intelliSenseMode": "msvc-x64",
    "mergeConfigurations": false,
    "compilerPath": "C:/Program Files (x86)/Microsoft Visual Studio/2019/Professional/VC/Tools/MSVC/14.27.29110/bin/Hostx64/x64/cl.exe",
    "browse": {
        "path": [
            "${workspaceFolder}/**",

        ]
    }
}
Translation Unit Mappings:
[ c:\workdir\CppFile.cpp ]:
    c:\workdir\CppFile.cpp
Translation Unit Configurations:
[ c:\workdir\CppFile.cpp ]: not ready
    Process ID: 23788
    Memory Usage: 221 MB
    Compiler Path: C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\VC\Tools\MSVC\14.27.29110\bin\Hostx64\x64\cl.exe
    Includes:
        U:\include
        V:\include
        C:\Program Files (x86)\Windows Kits\10\Include\10.0.17134.0\cppwinrt
        C:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\Include\um

    Defines:

    Standard Version: ms_c++20
    IntelliSense Mode: windows-msvc-x64
Total Memory Usage: 231 MB

------- Workspace parsing diagnostics -------
Number of files discovered (not excluded): 220379

### Other Extensions

_No response_

### Additional context

_No response_

[sean-mcmanus]

We're not reproing this. Can you provide more info? Is "workdir" your workspace folder? What file are you opening?

Also, we plan to release a 1.15.1 update tomorrow with some crash fixes, which may fix this issue (not sure without a call stack of the crashing cpptools process).

[psclkhoury]

@sean-mcmanus yes "workdir" is myworksapce folder. I open any cpp file to reproduce the problem.

I attached the debugger to cpptools.exe but there was no crash. However I noticed that in version 1.14.x the cpptools-srv.exe is not started. I even added flags to directly launch the vsjitdebugger when that process starts, but on 1.14.x it never does; on 1.13.9 I can successfully attach to it as soon as it's started.

[psclkhoury]

I managed to reproduce it with this:

The only thing needed to do to reproduce it is to have the folder on a virtual drive.

My test repo is on C:\Workdir\TestRepo and C:\Workdir is mapped to N:\. To open the folder I use code N:\TestRepo

Running subst will return:

N:\: => C:\Workdir

[daira]

I'm seeing similar crashes on Linux. In my case the project was opened in a subdirectory under a symlink to a mounted filesystem, which I guess could tickle the same bug as a virtual drive on Windows. Opening a copy of the same project on the main filesystem (containing my home directory) works. Then again, opening a different project under the symlink doesn't seem to give the same errors, so perhaps it's a combination of factors (or perhaps the symlink thing is a red herring).

[sean-mcmanus]

@psclkhoury The symptom of cpptools-srv not starting could be from the deadlock bug https://github.com/microsoft/vscode-cpptools/issues/10719 , which we plan to release a fix for later today. The crash with a virtual drive might be https://github.com/microsoft/vscode-cpptools/issues/10636 (I'll check later today).

@daira We don't currently support opening a folder path under a symlink on Linux see https://github.com/microsoft/vscode-cpptools/issues/4573 .

[sean-mcmanus]

I don't repro the issue with the network drive. Are you able to identify any other repro requirement that could trigger the crash? Or if you can attach a debugger to get a call stack for the cpptools process, that could help.

[sean-mcmanus]

We fixed some crashes and a deadlock with multiroot workspaces with 1.15.1 (https://github.com/microsoft/vscode-cpptools/releases/tag/v1.15.1), but it's possible you may be hitting a different crash.

[psclkhoury]

I tried 1.15.1 and I still have the crash. I attached the debugger to cpptools but there was not crash so the debugger did not stop to get a callstack.

I finally figured out what is causing the crash and it doesn't have to do with mapped drives.

The command promt that I was using to launch vscode had a weird Path environment variable that looked something like Path=c:\some\path;%PATH%;C:\some\other\path;... (notice the non-expanded %PATH% inside of it).

I can reproduce the crash with this C# console program.

using System;
using System.Diagnostics;

namespace CSharpConsole
{
    internal class Program
    {
        static void Main(string[] args)
        {
            Environment.SetEnvironmentVariable("PATH", string.Format("{0};{1}", "%PATH%", Environment.ExpandEnvironmentVariables("%PATH%")));
            Process.Start("code", "C:\\workdir\\testrepo");
        }
    }
}

[sean-mcmanus]

Fixed with https://github.com/microsoft/vscode-cpptools/releases/tag/v1.15.2 .

[psclkhoury]

fixed

[joao-moraes-petitfabrik]

I just updated to 1.15.2 but I am getting the same error, I am not sure if it's the same cause. What can I do or how can I help investigate the issue?

I am using Windows 10 x64

[Error - 3:03:48 PM] Sending document notification textDocument/didSave failed.
Error: Unsupported client
    at DefaultClient.<anonymous> (c:\Users\jvlppm\.vscode\extensions\ms-vscode.cpptools-1.15.2-win32-x64\dist\main.js:59155:23)
    at Generator.next (<anonymous>)
    at c:\Users\jvlppm\.vscode\extensions\ms-vscode.cpptools-1.15.2-win32-x64\dist\main.js:57549:71
    at new Promise (<anonymous>)
    at __webpack_modules__.9325.__awaiter (c:\Users\jvlppm\.vscode\extensions\ms-vscode.cpptools-1.15.2-win32-x64\dist\main.js:57545:12)
    at DefaultClient.queueTask (c:\Users\jvlppm\.vscode\extensions\ms-vscode.cpptools-1.15.2-win32-x64\dist\main.js:59132:16)
    at DefaultClient.requestWhenReady (c:\Users\jvlppm\.vscode\extensions\ms-vscode.cpptools-1.15.2-win32-x64\dist\main.js:59190:21)
    at invoke1 (c:\Users\jvlppm\.vscode\extensions\ms-vscode.cpptools-1.15.2-win32-x64\dist\main.js:65187:67)
    at DidSaveTextDocumentFeature.callback (c:\Users\jvlppm\.vscode\extensions\ms-vscode.cpptools-1.15.2-win32-x64\dist\main.js:82106:33)
    at c:\Users\jvlppm\.vscode\extensions\ms-vscode.cpptools-1.15.2-win32-x64\dist\main.js:82091:22
    at E.invoke (c:\Users\jvlppm\AppData\Local\Programs\Microsoft VS Code\resources\app\out\vs\workbench\api\node\extensionHostProcess.js:63:145)
    at h.deliver (c:\Users\jvlppm\AppData\Local\Programs\Microsoft VS Code\resources\app\out\vs\workbench\api\node\extensionHostProcess.js:63:2121)
    at n.fire (c:\Users\jvlppm\AppData\Local\Programs\Microsoft VS Code\resources\app\out\vs\workbench\api\node\extensionHostProcess.js:63:1729)
    at k.$acceptModelSaved (c:\Users\jvlppm\AppData\Local\Programs\Microsoft VS Code\resources\app\out\vs\workbench\api\node\extensionHostProcess.js:100:10783)
    at c.N (c:\Users\jvlppm\AppData\Local\Programs\Microsoft VS Code\resources\app\out\vs\workbench\api\node\extensionHostProcess.js:108:11655)
    at c.M (c:\Users\jvlppm\AppData\Local\Programs\Microsoft VS Code\resources\app\out\vs\workbench\api\node\extensionHostProcess.js:108:11373)
    at c.H (c:\Users\jvlppm\AppData\Local\Programs\Microsoft VS Code\resources\app\out\vs\workbench\api\node\extensionHostProcess.js:108:10454)
    at c.G (c:\Users\jvlppm\AppData\Local\Programs\Microsoft VS Code\resources\app\out\vs\workbench\api\node\extensionHostProcess.js:108:9432)
    at c:\Users\jvlppm\AppData\Local\Programs\Microsoft VS Code\resources\app\out\vs\workbench\api\node\extensionHostProcess.js:108:8220
    at E.invoke (c:\Users\jvlppm\AppData\Local\Programs\Microsoft VS Code\resources\app\out\vs\workbench\api\node\extensionHostProcess.js:63:145)
    at h.deliver (c:\Users\jvlppm\AppData\Local\Programs\Microsoft VS Code\resources\app\out\vs\workbench\api\node\extensionHostProcess.js:63:2121)
    at n.fire (c:\Users\jvlppm\AppData\Local\Programs\Microsoft VS Code\resources\app\out\vs\workbench\api\node\extensionHostProcess.js:63:1729)
    at p.fire (c:\Users\jvlppm\AppData\Local\Programs\Microsoft VS Code\resources\app\out\vs\workbench\api\node\extensionHostProcess.js:72:14916)
    at c:\Users\jvlppm\AppData\Local\Programs\Microsoft VS Code\resources\app\out\vs\workbench\api\node\extensionHostProcess.js:124:16654
    at E.invoke (c:\Users\jvlppm\AppData\Local\Programs\Microsoft VS Code\resources\app\out\vs\workbench\api\node\extensionHostProcess.js:63:145)
    at h.deliver (c:\Users\jvlppm\AppData\Local\Programs\Microsoft VS Code\resources\app\out\vs\workbench\api\node\extensionHostProcess.js:63:2121)
    at n.fire (c:\Users\jvlppm\AppData\Local\Programs\Microsoft VS Code\resources\app\out\vs\workbench\api\node\extensionHostProcess.js:63:1729)
    at p.fire (c:\Users\jvlppm\AppData\Local\Programs\Microsoft VS Code\resources\app\out\vs\workbench\api\node\extensionHostProcess.js:72:14916)
    at MessagePortMain.<anonymous> (c:\Users\jvlppm\AppData\Local\Programs\Microsoft VS Code\resources\app\out\vs\workbench\api\node\extensionHostProcess.js:124:14780)
    at MessagePortMain.emit (node:events:526:28)
    at MessagePortMain.emit (node:domain:475:12)
    at MessagePortMain._internalPort.emit (node:electron/js2c/utility_init:5:364)

[Colengms]

Hi @joao-moraes-petitfabrik Does it crash immediately on start-up? Or, would be there a way for you to attach the debugger to the native process (cpptools.exe) to capture the crash stack from that process? The JavaScript stack you posted is a later symptom of a native process crash. Also, can you confirm whether the issue is related to a 'recursive environment variable' in your environment? If not, and you're seeing a crash with a different cause, we could use a new issue to track it. We're very interested in crash reports and would appreciate any help you can provide.

https://github.com/microsoft/vscode-cpptools/wiki/Attaching-debugger-to-cpptools-or-cpptools%E2%80%90srv

[joao-moraes-petitfabrik]

Hi, sorry for the delay, it does take a while to crash, once I open my project the cpptools starts running and it seems it takes a long time to actualy load, because I have no code completion or anything for a few minutes, and then once it loads it crashes when I try to type anything.

I did attach to that process according to that guide, and it showed me a stack-trace

but it does not show the code associated, I´m not sure if that is enough to help, I hope it does.

I can do more tests, I can reproduce this issue easily on this project, but it is a large project that is under NDA so I can't share it.

[sean-mcmanus]

@joao-moraes-petitfabrik That is not a crash call stack -- it is throwing a C++ exception as part of normal operations (flow control) which is caught. You should disable the breaking on all exceptions so that it only breaks on the actual crash. The debugger output will show some message relating to a segmentation fault or unauthorized access exception, etc.