On Windows, none-ascii input filename will cause the filename to be truncated in log output if stderr is not tty

[sagan]

Environment:

flac-1.4.3 + Windows 10 / 11.

Description

If the input filename contains non-ASCII character, the stderr output of flac.exe will have the filename truncated before first non-ASCII character, if stderr is not tty (cmd / powershell terminal).

For example, set a wav filename to 01-声.wav, the 声 (A Kanji which means "sound") is a non-ASCII character.

Use flac.exe to convert this file to flac:

flac --force "01-声.wav"

If the command was executed in cmd / powershell, the output will be ok:

C:\tmp\a>flac --force "01-声.wav"

flac 1.4.3
Copyright (C) 2000-2009  Josh Coalson, 2011-2023  Xiph.Org Foundation
flac comes with ABSOLUTELY NO WARRANTY.  This is free software, and you are
welcome to redistribute it under certain conditions.  Type `flac' for details.

01-声.wav: wrote 98413 bytes, ratio=0.744

However, if the stderr was piped to file, or the flac.exe process is executed by other program, the output will have wrong filename:

flac --force "01-声.wav" 2>stderr.txt

stderr.txt:

flac 1.4.3
Copyright (C) 2000-2009  Josh Coalson, 2011-2023  Xiph.Org Foundation
flac comes with ABSOLUTELY NO WARRANTY.  This is free software, and you are
welcome to redistribute it under certain conditions.  Type `flac' for details.

01-wrote 98413 bytes, ratio=0.744

Futher more

If flac failed to process the input file, the whole error message will be lost in this case.

E.g. trying to convert a 32 float wav to flac:

flac --force "01 注意事項.wav"

Normal stderr output in cmd:

flac 1.4.3
Copyright (C) 2000-2009  Josh Coalson, 2011-2023  Xiph.Org Foundation
flac comes with ABSOLUTELY NO WARRANTY.  This is free software, and you are
welcome to redistribute it under certain conditions.  Type `flac' for details.

01 注意事項.wav: ERROR: unsupported format type 3

Output if stderr was piped to file

flac 1.4.3
Copyright (C) 2000-2009  Josh Coalson, 2011-2023  Xiph.Org Foundation
flac comes with ABSOLUTELY NO WARRANTY.  This is free software, and you are
welcome to redistribute it under certain conditions.  Type `flac' for details.

01 

[ktmf01]

Thanks for reporting, I will try to fix this.

[petterreinholdtsen]

[Martijn van Beurden]

Thanks for reporting, I will try to fix this.

Sound like the output is in UTF-16 or UTF-32, causing null bytes to show up in the string.

-- Happy hacking Petter Reinholdtsen

[ktmf01]

@sagan, are you able to compile the changes in #714 or would you like me to provide a compile? It would be great if you could check whether that fix works.

[sagan]

I tried to compile https://github.com/ktmf01/flac/tree/fix-utf8-output-redirection on Windows 11 using cmake + Visual Studio Community 2022

The code failed to comiple with an 'identifier "options" is undefined' error in flac/src/metaflac/operations.c. This can be fixed by change the signature of write_metadata_binary to pass utf8_convert to it as a arg, after which the build successes.

However, the generated flac.exe fails to process any input file with an assert error:

[ktmf01]

The code failed to comiple with an 'identifier "options" is undefined' error in flac/src/metaflac/operations.c. This can be fixed by change the signature of write_metadata_binary to pass utf8_convert to it as a arg, after which the build successes.

Oops. Apparently I didn't check that last change.

However, the generated flac.exe fails to process any input file with an assert error:

This seems linked, I will investigate: https://stackoverflow.com/questions/45575863/how-to-print-utf-8-strings-to-stdcout-on-windows

[ktmf01]

It was a little more difficult than I had anticipated, but I think #714 should now work properly. @sagan, could you test again for me?

[sagan]

I just compiled https://github.com/ktmf01/flac/tree/fix-utf8-output-redirection on Windows. It seems it's working properly now. The output file of redirected stderr is UTF-8 encoding and have non-ASCII characters stored correctly.

Thank you!