Windows平台下C++项目使用lld-link时传递参数字符串编码为utf-8导致链接参数无法被linker识别

[luciouskami]

Xmake 版本

v2.9.4+HEAD.e85b001f1

操作系统版本和架构

windows 11 23H2

描述问题

如题，Windows平台下C++项目使用lld-link时传递参数字符串编码为utf-8，链接参数无法被linker识别。经过测试，手动复制编译命令使用ansi的情况下可以正常链接，但是单独的demo很难复现，这个要在长路径+多中文文件名的情况下才能复现，提供demo有些困难。在开启windows 11 的区域设置将系统的默认编码设置为utf-8时链接正常，但是系统开启utf-8会对个别应用造成影响，不是很好的解决方案。

期待的结果

传递链接参数时使用本地编码，使得不开启区域设置将系统的默认编码设置为utf-8时链接也能正常工作

工程配置

无

附加信息和错误日志

lld-link: error: could not open 'build\.objs\server\windows\x86\release\src\2.server\xxx\xxx\中文.cpp.obj': 在多字节的目标代码页中，没有此 Unicode 字符可以映射到的字符。

[Issues-translate-bot]

Bot detected the issue body's language is not English, translate it automatically.

---

Title: When using lld-link in a C++ project on the Windows platform, the parameter string passed is encoded as utf-8, resulting in the link parameters not being recognized by the linker.

Xmake version

v2.9.4+HEAD.e85b001f1

Operating system version and architecture

Windows 11 23H2

Describe the problem

As the title states, when using lld-link in a C++ project on the Windows platform, the parameter string passed is encoded as utf-8, and the link parameters cannot be recognized by the linker. After testing, it can be linked normally when using ansi, but it is difficult to reproduce the demo alone. This can only be reproduced when there are long paths + multiple Chinese file names. It is somewhat difficult to provide demos. When the regional settings of Windows 11 are turned on and the system's default encoding is set to utf-8, the link is normal. However, turning on utf-8 in the system will affect individual applications, which is not a good solution.

Expected results

Use local encoding when passing link parameters so that the link can work normally even if the regional settings are not turned on and the system's default encoding is set to utf-8.

Project configuration

none

Additional information and error logs

None

[waruqi]

这个要在长路径+多中文文件名的情况下才能复现

如果是长路径，其实现在已经是 ansi 了。。长路径会走 file list 作为 args，不走终端命令行。

https://github.com/xmake-io/xmake/blob/2f3f1a02d2e28f77e1da4a7291fe01a0642f2a74/xmake/modules/core/tools/link.lua#L133

https://github.com/xmake-io/xmake/blob/2f3f1a02d2e28f77e1da4a7291fe01a0642f2a74/xmake/core/base/winos.lua#L171

[luciouskami]

这个要在长路径+多中文文件名的情况下才能复现

如果是长路径，其实现在已经是 ansi 了。。长路径会走 file list 作为 args，不走终端命令行。

https://github.com/xmake-io/xmake/blob/2f3f1a02d2e28f77e1da4a7291fe01a0642f2a74/xmake/modules/core/tools/link.lua#L133

https://github.com/xmake-io/xmake/blob/2f3f1a02d2e28f77e1da4a7291fe01a0642f2a74/xmake/core/base/winos.lua#L171

那就有点迷了，这个报错，编码这个问题，因为用命令行手动输入链接命令的话确实是能过的

[luciouskami]

我把ansi改成utf-8，它竟然好了，可能要探测一下看看原本输入的编码是什么，否则出现非英文字符可能会出问题

[Issues-translate-bot]

Bot detected the issue body's language is not English, translate it automatically.

---

This can only be reproduced if there are long paths + multiple Chinese file names.

If it is a long path, it is actually ansi now. . For long paths, file list will be used as args instead of the terminal command line.

https://github.com/xmake-io/xmake/blob/2f3f1a02d2e28f77e1da4a7291fe01a0642f2a74/xmake/modules/core/tools/link.lua#L133

https://github.com/xmake-io/xmake/blob/2f3f1a02d2e28f77e1da4a7291fe01a0642f2a74/xmake/core/base/winos.lua#L171

[Issues-translate-bot]

Bot detected the issue body's language is not English, translate it automatically.

---

This can only be reproduced if there are long paths + multiple Chinese file names.

If it is a long path, it is actually ansi now. . For long paths, file list will be used as args instead of the terminal command line.

https://github.com/xmake-io/xmake/blob/2f3f1a02d2e28f77e1da4a7291fe01a0642f2a74/xmake/modules/core/tools/link.lua#L133

https://github.com/xmake-io/xmake/blob/2f3f1a02d2e28f77e1da4a7291fe01a0642f2a74/xmake/core/base/winos.lua#L171

Then I’m a little confused. How should I test this error? As for the encoding problem, it is indeed possible to pass the link command manually by using the command line.

[Issues-translate-bot]

Bot detected the issue body's language is not English, translate it automatically.

---

I changed ansi to utf-8 and it worked. I may need to adjust it to see what the original input encoding was.

[waruqi]

我把ansi改成utf-8，它竟然好了，可能要探测一下看看原本输入的编码是什么，否则出现非英文字符可能会出问题

xmake 内部所有字符串都是 utf8，除非是写文件 显式指定了 ansi 。。

[luciouskami]

我把ansi改成utf-8，它竟然好了，可能要探测一下看看原本输入的编码是什么，否则出现非英文字符可能会出问题

xmake 内部所有字符串都是 utf8，除非是写文件 显式指定了 ansi 。。

那这个问题应该怎么解决比较好呢？理论上windows命令行程序输入的参数的编码是是ansi的，但是这里又不是。

[Issues-translate-bot]

Bot detected the issue body's language is not English, translate it automatically.

---

I changed ansi to utf-8, and it turned out fine. You may need to check to see what the original input encoding was, otherwise problems may arise if non-English characters appear.

All strings inside xmake are utf8, unless ansi is explicitly specified when writing a file. .

[Issues-translate-bot]

Bot detected the issue body's language is not English, translate it automatically.

---

I changed ansi to utf-8, and it worked. I may need to check to see what the original input encoding was, otherwise there may be problems if non-English characters appear.

All strings inside xmake are utf8, unless ansi is explicitly specified when writing a file. .

So what is the best way to solve this problem? Theoretically, the encoding of parameters input into the Windows command line program is ANSI, but it is not the case here.

[waruqi]

不知道，我这边也没换环境复现。。先确认是否只有 lld-link.exe 支持 utf8 不支持 ansi，如果只是这一个特例，那就只能针对它单独处理。

[Issues-translate-bot]

Bot detected the issue body's language is not English, translate it automatically.

---

I don’t know, I haven’t changed the environment to reproduce it. . First confirm whether only lld-link.exe supports utf8 and not ansi. If it is just a special case, it can only be dealt with separately.

[luciouskami]

不知道，我这边也没换环境复现。。先确认是否只有 lld-link.exe 支持 utf8 不支持 ansi，如果只是这一个特例，那就只能针对它单独处理。

测试下来，在控制台本地编码是936的情况下手动输入链接命令能正常链接，在xmake中指定ansi不行，反而使用utf-8却可以，这就是说不通的地方。

[Issues-translate-bot]

Bot detected the issue body's language is not English, translate it automatically.

---

I don’t know, I haven’t changed the environment to reproduce it. . First confirm whether only lld-link.exe supports utf8 and not ansi. If it is just a special case, it can only be dealt with separately.

After testing, when the local encoding of the console is 936, manually entering the link command can link normally. Specifying ansi in xmake does not work, but using utf-8 does. This is where it doesn't make sense.