facebook / buck2

Build system, successor to Buck
https://buck2.build/
Apache License 2.0
3.33k stars 194 forks source link

Support multiple execution platforms with system_cxx_toolchain #629

Open Overhatted opened 2 weeks ago

Overhatted commented 2 weeks ago

As mentioned in #612, I have a C++ project that can be built for either Linux or Windows, without any cross-compilation so Linux and Windows are also the two execution platforms. Since there is remote execution, I wanted to be able to build for Linux from my Windows machine using RE, and vice versa.

From the response I got in the above issue, I need to add an exec_dep to the toolchain, pointing a compiler target, that then has a target_compatible_with with the execution platform the compiler can run on.

I tried to add the target_compatible_with but it didn't work, probably intentional.

So I moved some parts of the current system_cxx_toolchain to a separate NativeCompiler provider that is then returned by some rules defining a compiler. This compiler target then has the target_compatible_with attribute.

Can this be merged or should I just have this toolchain in my own repository? It seems useful for everyone, the only disadvantage I can see being that it is not fully backwards compatible. Hopefully I didn't completely misunderstand how execution platforms are supposed to be used.

I improved the Visual Studio example to use this new updated toolchain to easily support remote execution.

Manual testing

I used the Visual Studio example and setup my RE workers with Buildbarn. Unfortunately its Windows workers don't really work and the linking step fails with "The system cannot open the device or file specified.". I haven't investigated why.

Local builds

Windows

buck2 build --target-platforms //buck2_utils/platforms:windows_debug :main

Passes

Linux

buck2 build --target-platforms //buck2_utils/platforms:linux_debug :main

Passes

Remote builds

From Windows

Target Windows
buck2 build --no-remote-cache -c buck2_re_client.enabled=true --target-platforms //buck2_utils/platforms:windows_debug :main

Linking fails (The system cannot open the device or file specified.)

Target Linux
buck2 build --no-remote-cache -c buck2_re_client.enabled=true --target-platforms //buck2_utils/platforms:linux_debug :main

Passes

From Linux

Target Windows
buck2 build --no-remote-cache -c buck2_re_client.enabled=true --target-platforms //buck2_utils/platforms:windows_debug :main

Linking fails (The system cannot open the device or file specified.)

Target Linux
buck2 build --no-remote-cache -c buck2_re_client.enabled=true --target-platforms //buck2_utils/platforms:linux_debug :main

Passes