Garbled output at PowerShell when I use redirect.

frg2089 commented 1 year ago

Describe the bug

When I use PowerShell in a Chinese environment to redirect the output of a dotnet program, I get some garbled.

It can also happen in the pipeline.

I tested MSBuild for Visual Studio, MSBuild for .Net Framework and other applications. Only dotnet does this.

And I tested PowerShell 7.3, Windows PowerShell 5.1 and Command Prompt. this only happens on PowerShell 7.3 and Windows PowerShell 5.1.

To Reproduce

Set your Windows Display Language is Chinese.

dotnet -? > stdout.log # Redirect standard output into stdout.log file.
# or
dotnet -? | clip # Copy the standard output into your clipboard.

Further technical details

.NET SDK:
 Version:   7.0.203
 Commit:    5b005c19f5

运行时环境:
 OS Name:     Windows
 OS Version:  10.0.22621
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\7.0.203\

Host:
  Version:      7.0.5
  Architecture: x64
  Commit:       8042d61b17

.NET SDKs installed:
  6.0.406 [C:\Program Files\dotnet\sdk]
  7.0.203 [C:\Program Files\dotnet\sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 6.0.14 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 6.0.16 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 7.0.5 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 6.0.14 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 6.0.16 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 7.0.5 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.WindowsDesktop.App 6.0.14 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 6.0.16 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 7.0.5 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

Other architectures found:
  x86   [C:\Program Files (x86)\dotnet]
    registered at [HKLM\SOFTWARE\dotnet\Setup\InstalledVersions\x86\InstallLocation]

Environment variables:
  Not set

global.json file:
  Not found

Learn more:
  https://aka.ms/dotnet/info

Download .NET:
  https://aka.ms/dotnet/download

Windows Display Language

Direct run in PowerShell 7.3

Redirect into file

dotnet-issue-labeler[bot] commented 1 year ago

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

frg2089 commented 1 year ago

It also in .Net 8.0

frg2089 commented 1 year ago

This garble looks like it is caused by UTF-8 encoded content being saved incorrectly as GB18030 (GBK or GB2312).

frg2089 commented 1 year ago

Maybe it's a problem here.

https://github.com/dotnet/sdk/blob/abc94993c152aa557f953b897abaaf38dff61f86/src/Cli/dotnet/Program.cs#L33-L41

It looks like the code here causes being redirected when the current OS is not iOS, Android, TV OS and browser to force UTF-8 encoding, while PowerShell running on Windows will treat them as if they were ANSI.

s1iver1ining commented 5 months ago

yes, it IS because the encoding issues, but i don't know how to solve it. My problem is using redirection in cmd prompt when I was learning chapter8. But both my input and output don't have ANY Chinese character, but it still went wrong. Two lines were replaced by garbled codes! For No reason!! This is just like a black box, and I have nothing to do about it, so powerless... 3-14

frg2089 commented 5 months ago

yes, it IS because the encoding issues, but i don't know how to solve it. My problem is using redirection in cmd prompt when I was learning chapter8. But both my input and output don't have ANY Chinese character, but it still went wrong. Two lines were replaced by garbled codes! For No reason!! This is just like a black box, and I have nothing to do about it, so powerless...

What is your dotnet version? and What's your PowerShell version?

dotnet / sdk