search

mstorsjo / llvm-mingw

An LLVM/Clang/LLD based mingw-w64 toolchain
Other
1.97k stars 188 forks source link

[suggestion]use configuration file replace clang wrapper #253

Open SquallATF opened 2 years ago

SquallATF commented 2 years ago

via https://clang.llvm.org/docs/UsersManual.html#configuration-files Using configuration file can reduce one process invoke but no ccache support, clang clang++ gcc g++ cc c++ can support configuration file but c99 c11 as can not.

https://reviews.llvm.org/D53066 is needed for windows.

here is a example for i686-w64-mingw32 and i686-w64-mingw32uwp.

1. Put the following 4 files in the bin directory 

-rtlib=compiler-rt
-unwindlib=libunwind
-stdlib=libc++
-fuse-ld=lld
-Qunused-arguments
-D_WIN32_WINNT=0x0A00
-DWINVER=0x0A00
-DWINAPI_FAMILY=WINAPI_FAMILY_APP
-DUNICODE
-Wl,-lwindowsapp
-Wl,-lucrtapp
-D_UCRT
-target i686-w64-mingw32
@mingw32-common.opts
@mingw32-uwp.opts
-target i686-w64-mingw32
@mingw32-common.opts

2. Create i686-w64-mingw32-gcc.exe link to clang.exe, when run i686-w64-mingw32-gcc.exe will auto load i686-w64-mingw32.cfg, and create i686-w64-mingw32uwp-gcc.exe link to clang.exe, when run i686-w64-mingw32-gcc.exe will auto load i686-w64-mingw32uwp.cfg, 

$ /e/llvm13/bin/i686-w64-mingw32-gcc.exe -v
clang version 13.0.0 (https://github.com/llvm/llvm-project.git ecf14384e1355a0b498e704d268acb8e60826a28)
Target: i686-w64-windows-gnu
Thread model: posix
InstalledDir: E:\llvm13\bin
Configuration file: E:\llvm13\bin\i686-w64-mingw32.cfg
$ /e/llvm13/bin/i686-w64-mingw32uwp-gcc.exe -v
clang version 13.0.0 (https://github.com/llvm/llvm-project.git ecf14384e1355a0b498e704d268acb8e60826a28)
Target: i686-w64-windows-gnu
Thread model: posix
InstalledDir: E:\llvm13\bin
Configuration file: E:\llvm13\bin\i686-w64-mingw32uwp.cfg
mstorsjo commented 2 years ago

Hi, thanks for the suggestion!

via https://clang.llvm.org/docs/UsersManual.html#configuration-files Using configuration file

Yup, I've actually experimented with using this a couple years ago already. (I have a PoC branch locally, but it's quite some time since I last rebased and tried it out.) I think the main stumbling block was that it requires symlinks (which I guess are more or less supported in practice on Windows these days, or is that only as administrator and/or if the computer is set up in developer mode? - and they don't work through zip files as I use for distribution, as I want it to be unpackable out of the box without any other tools).

Without symlinks, previously it would have caused huge duplication due to making a new copy of a 90 MB clang.exe for each of the frontends. But since switching to building with dylibs in 46efff459a2665bb92d5aac407f23949b23f65fd, the main clang-14.exe is only 75-110 KB or thereabout, so maybe it could even be tolerable? (The clang-target-wrapper.exe which is duplicated today is 17 KB.)

can reduce one process invoke

Do you have any measurement on whether this does affect the total compilation time in some scenario, and how much? I would guess that it has a bigger impact on Windows than on unix, but I'm still curious about how big the impact is.

but no ccache support,

That shouldn't be an issue, I don't really use that feature in practice

clang clang++ gcc g++ cc c++ can support configuration file but c99 c11 as can not.

Ok - I guess I can try to fix upstream so that it would recognize c99/c11 too (not sure if upstream would accept handling as as suffix too though).

https://reviews.llvm.org/D53066 is needed for windows.

This isn't needed any longer; I've fixed this properly upstream now, which will be part of the 14.0.0 release, see https://github.com/llvm/llvm-project/commits/df0ba47c36f6bd0865e3286853b76d37e037c2d7. I pushed a branch next to this repo, which includes a bunch of cleanups that I can include once I bump build-llvm.sh to point to a newer version (probably going to point it at 14.0.0rc1 when that's out), which includes this commit: https://github.com/mstorsjo/llvm-mingw/commit/d4bcca7be5df1b9d66736bcaf0376ee53f6ded1f

here is a example for i686-w64-mingw32 and i686-w64-mingw32uwp.

1. Put the following 4 files in the bin directory

  • mingw32-common.opts
-rtlib=compiler-rt
-unwindlib=libunwind
-stdlib=libc++
-fuse-ld=lld
-Qunused-arguments

FWIW, -Qunused-arguments isn't needed when using config files. It's only needed when specifying arguments on the command line (as it otherwise warns about e.g. -stdlib when compiling C code). When using config files, Clang specifically ignores warning about unused arguments from the config files. I also added another way of handling that upstream recently, see https://github.com/llvm/llvm-project/commit/50ec1306d060e46e0d53c9f5d8a052e1b0d10d3b, which lets me do this: https://github.com/mstorsjo/llvm-mingw/commit/5ba36f4cf83298f9719f0d2b561e539435390f6e That lets the toolchain still warn about unused arguments that the user has specified.

  • mingw32-uwp.opts
-D_WIN32_WINNT=0x0A00
-DWINVER=0x0A00
-DWINAPI_FAMILY=WINAPI_FAMILY_APP
-DUNICODE
-Wl,-lwindowsapp
-Wl,-lucrtapp
-D_UCRT
  • i686-w64-mingw32uwp.cfg
-target i686-w64-mingw32
@mingw32-common.opts
@mingw32-uwp.opts
  • i686-w64-mingw32.cfg
-target i686-w64-mingw32
@mingw32-common.opts

Thanks, the use of @<foo> here for splitting things up is useful, I don't think I had tried that in my older PoC for using config files.

nolange commented 3 months ago

I experimented with this as well, I used the lookup strategy to separate configs for compiler-type and target. UWP would require a different triple (armv7-w64-windows-gnuuwp-clangfor example).

Otherwise, symlinks to clang with the current names should work.

(-resource-dir is purely for using these configs with a system clang)

clang-cpp.cfg

# Allow other clang installations to use this config
-resource-dir="<CFGDIR>/../lib/clang/18"

clang.cfg

# Allow other clang installations to use this config
-resource-dir="<CFGDIR>/../lib/clang/18"

-rtlib=compiler-rt
-unwindlib=libunwind
-fuse-ld=lld

clang++.cfg

# Allow other clang installations to use this config
-resource-dir="<CFGDIR>/../lib/clang/18"

-rtlib=compiler-rt
-unwindlib=libunwind
-fuse-ld=lld

-stdlib=libc++

armv7-w64-windows-gnu.cfg

--sysroot="<CFGDIR>/../lib/armv7-w64-mingw32"

armv7-w64-windows-gnuuwp.cfg

--sysroot="<CFGDIR>/../lib/armv7-w64-mingw32"

-D_WIN32_WINNT=0x0A00
-DWINVER=0x0A00
-DWINAPI_FAMILY=WINAPI_FAMILY_APP
-DUNICODE
-D_UCRT

--start-no-unused-arguments
-Xlinker -lwindowsapp
-Xlinker -lucrtapp
--end-no-unused-arguments
mstorsjo commented 1 month ago

FYI, I'm seriously considering taking config files into use - I have a branch prepared for this now, see https://github.com/mstorsjo/llvm-mingw/commits/clang-cfg, and am currently planning on merging this before the next point release in 2 weeks.

After fixing a couple of issues around clang-scan-deps, that tool can now also pick up defaults implicitly from config files, so the clang-scan-deps wrapper can be avoided entirely on unix - CC @huangqinjin.

By skipping the hardcoded defaults in the clang binary on Windows (which never was strictly needed when we use wrappers anyway), we also ease use of the clangd binary with other mingw toolchains, see https://github.com/mstorsjo/llvm-mingw/issues/454#issuecomment-2364584265. Also CC @alvinhochun, even if this doesn't affect your case in #444.

mstorsjo commented 4 weeks ago

FYI, I'm seriously considering taking config files into use - I have a branch prepared for this now, see https://github.com/mstorsjo/llvm-mingw/commits/clang-cfg, and am currently planning on merging this before the next point release in 2 weeks.

I did try merging this, but it uncovered two issues - see https://github.com/mstorsjo/llvm-mingw/commit/e0964ce24c086bb9f7de0916157f9177e85049d6 for references around that.

Will try to reapply these changes once we have those two issues sorted out.