xmake-io / xmake

🔥 A cross-platform build utility based on Lua
https://xmake.io
Apache License 2.0
9.89k stars 775 forks source link

Set compiler's input encoding #2471

Closed VlaDexa closed 1 year ago

VlaDexa commented 2 years ago

Is your feature request related to a problem? Please describe.

I was messing around with some non-ASCII text conversion and found out that MSVC need an extra flag to suppor UTF-8 inputs. After further investigation I found that different compilers like GCC and MSVC use different options for such tasks (because of course they do), so it would be neat if XMake could detect these flags or even have a different set_ command for these tasks

Describe the solution you'd like

Adding a command like set_encoding("UTF-8") would cause XMake to omit compiler options such as /utf8 for msvc or -fexec-charset=UTF-8 -fextended-identifiers -finput-charset=UTF-8 for GCC

I think clang by default supports UTF-8 (needs further research)

Describe alternatives you've considered

Including these options into compiler options mappings, i guess?

Additional context

No response

waruqi commented 2 years ago

you can try

if is_plat("windows") then
    add_cxflags("/utf8")
else
    add_cxflags("-fexec-charset=UTF-8", "-fextended-identifiers", "-finput-charset=UTF-8")
end

This does not seem to require complex configuration, and xmake will also automatically ignore these flags if they are not supported by the current compiler.

I need to think carefully about adding new interfaces, because I don't want to add too many unless it's a very necessary interface.

Of course, if more people feel that adding set_encodings is necessary, then I'll consider it as well.

Winterreisender commented 1 year ago

This is a necessary and useful feature implemented by many well-known build systems.

For examples, in Maven, a new config file always comes with these lines:

<properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <project.build.targetEncoding>UTF-8</project.build.targetEncoding>
</properties>

In Gradle, there's

options.encoding = "UTF-8"
waruqi commented 1 year ago

I have supported it. xmake update -s dev

https://github.com/xmake-io/xmake/pull/4019

-- for all source/target encodings
set_encodings("utf-8") -- msvc: /utf-8
set_encodings("source:utf-8", "target:utf-8")

-- gcc/clang: -finput-charset=UTF-8, msvc: -source-charset=utf-8
set_encodings("source:utf-8")

-- gcc/clang: -fexec-charset=UTF-8, msvc: -target-charset=utf-8
set_encodings("target:utf-8")