[LiveLogger] Garbled output in cmd.exe

[ladipro]

Issue Description

LiveLogger does not work in cmd.exe.

Steps to Reproduce

dotnet build /ll in cmd.exe.

Expected Behavior

Either LiveLogger errors out or MSBuild falls back to regular console logging.

Actual Behavior

My first attempt to use LiveLogger resulted in this:

Analysis

If this console is not fancy-capable then the logger should not be allowed to run. Note that you still get this kind of console when running the VS Developer prompt as administrator so this would be a blocker for enabling LL by default.

Versions & Configurations

No response

[KalleOlaviNiemitalo]

MSBuild has code for setting the ENABLE_VIRTUAL_TERMINAL_PROCESSING mode:

https://github.com/dotnet/msbuild/blob/dfd8f413a80cd0865f968b2c0ad9b09c0df8c430/src/Build/BackEnd/Client/MSBuildClient.cs#L392-L394

I suppose this is not executed in the LiveLogger scenario, or is executed by a process that uses a different console.

[rainersigwald]

I would have expected the livelogger to not be enabled in this case

https://github.com/dotnet/msbuild/blob/dfd8f413a80cd0865f968b2c0ad9b09c0df8c430/src/MSBuild/XMake.cs#L3436-L3455

So that's likely the first thing to chase.

[baronfel]

@ladipro do you run cmd inside Windows Terminal? if so the WT_SESSION would be non-empty.

[ladipro]

@baronfel I use the "Developer Command Prompt for VS 2022" shortcut created by VS installer. It has %comspec% /k "C:\Program Files\Microsoft Visual Studio\2022\Preview\Common7\Tools\VsDevCmd.bat" as its target. It does not run cmd inside Windows Terminal, as far as I can tell. The WT_SESSION environment variable is not defined.

Ah, not defined! The fix is to check for null in addition to "" in the code @rainersigwald linked above.

[baronfel]

Oh, that makes sense too :+1: I did a check real quick and verified that cmd inside Windows Terminal does render correctly (which I hoped since Windows Terminal was doing the rendering portion).