Error: Command line contains characters that are not supported in the active code page (1252)

[ni4]

Recently our CI (for RNP OpenPGP implementation), which is run via the Github Actions, and worked well for years, started to fail on tests which checks unicode command line arguments. While the error message is pretty obvious and related to codepage, I was unable to find out what's going wrong (given that things worked perfectly). Code is run via msys shell, i.e. msys2 {}.

Did the following checks:

• windows runner codepage via chcp: 65001
• msys2 shell codepage via chcp.com: 65001
• printing unicode characters via the echo -e "\xE2\x9C\x93": works well (prints✓`).
• code, which check for unicode input, is run via the CreateProcessW() with (LPWSTR) cmdline_ws.c_str() parameter, defined as std::wstring cmdline_ws.
• LANG=en_US.UTF-8

Does anybody have any suggestion what could go wrong, or at least where is the correct place to ask this question? Thanks!

[andytygv]

Same problem with ffmpeg built with latest media autobuild suite. A search for this string indicates that it appeared on any MSYS2 binaries after 20240727 release, eg. /lib/crt1.o, ...

[jeremyd2019]

I just saw a reference to this error on the mingw-w64 mailing list, and I thought I saw a mention of it before, turns out it was here. https://sourceforge.net/p/mingw-w64/mailman/message/58839241/.

mingw-w64/mingw-w64@0d42217123d3aec0341b79f6d959c76e09648a1e

[jeremyd2019]

• windows runner codepage via chcp: 65001
• msys2 shell codepage via chcp.com: 65001

Those set the console codepage. The code added to mingw-w64 uses the ANSI codepage, which I don't think has a handy way to switch like the console codepage. There was a reference to manifesting for UTF-8 that might work for you, if you are running on a new enough Windows version.

Does anybody have any suggestion what could go wrong, or at least where is the correct place to ask this question? Thanks!

mingw-w64-public@lists.sourceforge.net

[andytygv]

manifesting for utf-8 does work for me (windows 11). thank you!

[ni4]

@jeremyd2019 Thanks for the links! That explains everything. The default system codepage is indeed 1252 so that's why it is used.

[rhaberkorn]

Before somebody spends hours to solve this: You can bypass the new checks as well by linking with -municode and defining wmain(). From there, you either use argv or call GetCommandLineW(). I am personally using glib's g_win32_get_command_line() which already returns UTF-8. This should work with older versions of Windows, as opposed to the proposed "manifest" solution.

The nasty thing about the runtime change is that it broke apps that weren't actually suffering from any security issues and were handling Unicode command lines just fine.

[jeremyd2019]

I'd take any concerns about this change to mingw-w64-public@lists.sourceforge.net.

[jeremyd2019]

The commit which added this error has been reverted in https://github.com/mingw-w64/mingw-w64/commit/c6857dc97a70e5587e639a107255b0d2d1e7a200, it seems it's going to get some reconsideration. Perhaps after mingw-w64 is updated in this repo this issue can be closed?

[ni4]

@jeremyd2019 I don't mind closing this, as all the questions are answered, and working solutions are posted. And it still would be searchable for others who could get into this issue. Again, thanks for help in solving this!

[jeremyd2019]

Revert commit included in #22512