facebook / buck2

Build system, successor to Buck
https://buck2.build/
Apache License 2.0
3.37k stars 199 forks source link

Interested in adding toolchain support for MSVC #163

Open zjturner opened 1 year ago

zjturner commented 1 year ago

I don't see any toolchain support for MSVC. The documentation is light in this area, so it's not even clear what the low level anatomy of a toolchain file even is. I've looked at the zig toolchain for inspiration, but there's a lot of magic syntax in the files that isn't really explained anywhere.

I think it would help to have a guide "How to add support for a new toolchain" that describes in detail what all the various commands do, overall syntax rules for the file (it looks like python, but not quite), and anything else that someone with no experience with bazel or buck might need in order to get started.

In the meantime, what's the best place to start poking if I want to try to get MSVC support working?

steveklabnik commented 1 year ago

Check this out: https://github.com/facebook/buck2/issues/141#issuecomment-1504650390

lmvasquezg commented 1 year ago

Great time to raise this up. MSVC support is on the way as we speak, we've been modifying out cxx rules which are quite clang centric to also support the MSVC toolchain.

Zig is a very complete example of what you want a toolchain to look like.

For MSVC specifically I recently added support for preprocessing and compilation. The minimum requirements for an MSVC toolchain would be something like this (modifying our c++ hello world example)

system_cxx_toolchain(
    name = "cxx",
    visibility = ["PUBLIC"],
    compiler = "windows",
    cxx_compiler = "cl",
    cpp_dep_tracking_mode = "show_includes",
    compiler_type = "windows"
)

This, after adding cpp_dep_tracking_mode and compiler_type as attrs to system_cxx_toolchain given that now they default to "makefile" and "clang" respectively as system_cxx_toolchain is a very simple abstraction of what our toolchains looks like.

However, the issues explained in this thread affect MSVC aswell, but we're working on improving this and I'll personally add a full MSVC example when all support is done and tested.

there's a lot of magic syntax in the files that isn't really explained anywhere.

Could you elaborate on this so we can improve out docs? Any specific line/field?

Thank you for your interest and we'd be glad for you to try this out and give us some feedback once is completed.

steveklabnik commented 1 year ago

I'll personally add a full MSVC example when all support is done and tested.

If I can do anything to assist, please help me know!

zjturner commented 1 year ago

there's a lot of magic syntax in the files that isn't really explained anywhere.

Could you elaborate on this so we can improve out docs? Any specific line/field?

Here's a few illustrative examples of the kind of thing that is missing.

Consider this example from the Getting Started Guide

image

Other questions are like:

I'll probably end up with more questions as I play around, this is just my initial pass, as someone coming to this having never used buck1 or bazel.

steveklabnik commented 1 year ago

As an update to my MSVC workarounds, my "hello world" works with this patch:

diff --git a/toolchains/rust.bzl b/toolchains/rust.bzl
index 9315fce..cd8b790 100644
--- a/toolchains/rust.bzl
+++ b/toolchains/rust.bzl
@@ -23,7 +23,7 @@ _DEFAULT_TRIPLE = select({
         # default when we're able; but for now buck2 doesn't work with the MSVC
         # toolchain yet.
         "config//cpu:arm64": "aarch64-pc-windows-gnu",
-        "config//cpu:x86_64": "x86_64-pc-windows-gnu",
+        "config//cpu:x86_64": "x86_64-pc-windows-msvc",
     }),
 })

and configuring the cxx toolchain like this:

system_cxx_toolchain(
    name = "cxx",
    linker = "link.exe",
    visibility = ["PUBLIC"],
)

this obviously is very hacky and would have issues with more complex things, but it does let me do work for now.

steveklabnik commented 1 year ago

Glad to see this default change! I am still running into some odd MSVC issues that are surfacing now that I'm doing even slightly more complex projects. When using @dtolnay's latest build script runner, I am getting linker errors again:

Action failed: root//third-party:semver-1.0.17-build-script-build (rustc bin-pic-static_pic-link/build_script_build-link bin,pic,link [diag])
Local command returned non-zero exit code 1
Reproduce locally: `"buck-out\\v2\\gen\\prelude\\fb50fd37ce946800\\python_bootstrap\\tools\\__win_python_wrapper__\\win_ ...<omitted>... y\\__semver-1.0.17-build-script-build__\\bin-pic-static_pic-link\\build_script_build-link-diag.args" (run `buck2 log what-failed` to get the full command)`
stdout:
stderr:
error: linking with `buck-out\v2\gen\root\fb50fd37ce946800\third-party\__semver-1.0.17-build-script-build__\__linker_wrapper.bat` failed: exit code: 1181
  |
  = note: "cmd" "/c" "buck-out\\v2\\gen\\root\\fb50fd37ce946800\\third-party\\__semver-1.0.17-build-script-build__\\__linker_wrapper.bat" "/NOLOGO" "C:\\Users\\steve\\Documents\\GitHub\\buck-rust-hello\\buck-out\\v2\\tmp\\root\\fb50fd37ce946800\\third-party\\__semver-1.0.17-build-script-build__\\rustc\\_buck_871788e481b8cd96\\rustcqWXLE0\\symbols.o" "buck-out\\v2\\gen\\root\\fb50fd37ce946800\\third-party\\__semver-1.0.17-build-script-build__\\bin-pic-static_pic-link\\extras\\build_script_build\\build_script_build.build_script_build.295b7d9c-cgu.0.rcgu.o" "buck-out\\v2\\gen\\root\\fb50fd37ce946800\\third-party\\__semver-1.0.17-build-script-build__\\bin-pic-static_pic-link\\extras\\build_script_build\\build_script_build.build_script_build.295b7d9c-cgu.1.rcgu.o" "buck-out\\v2\\gen\\root\\fb50fd37ce946800\\third-party\\__semver-1.0.17-build-script-build__\\bin-pic-static_pic-link\\extras\\build_script_build\\build_script_build.build_script_build.295b7d9c-cgu.10.rcgu.o" "buck-out\\v2\\gen\\root\\fb50fd37ce946800\\third-party\\__semver-1.0.17-build-script-build__\\bin-pic-static_pic-link\\extras\\build_script_build\\build_script_build.build_script_build.295b7d9c-cgu.11.rcgu.o" "buck-out\\v2\\gen\\root\\fb50fd37ce946800\\third-party\\__semver-1.0.17-build-script-build__\\bin-pic-static_pic-link\\extras\\build_script_build\\build_script_build.build_script_build.295b7d9c-cgu.12.rcgu.o" "buck-out\\v2\\gen\\root\\fb50fd37ce946800\\third-party\\__semver-1.0.17-build-script-build__\\bin-pic-static_pic-link\\extras\\build_script_build\\build_script_build.build_script_build.295b7d9c-cgu.13.rcgu.o" "buck-out\\v2\\gen\\root\\fb50fd37ce946800\\third-party\\__semver-1.0.17-build-script-build__\\bin-pic-static_pic-link\\extras\\build_script_build\\build_script_build.build_script_build.295b7d9c-cgu.14.rcgu.o" "buck-out\\v2\\gen\\root\\fb50fd37ce946800\\third-party\\__semver-1.0.17-build-script-build__\\bin-pic-static_pic-link\\extras\\build_script_build\\build_script_build.build_script_build.295b7d9c-cgu.15.rcgu.o" "buck-out\\v2\\gen\\root\\fb50fd37ce946800\\third-party\\__semver-1.0.17-build-script-build__\\bin-pic-static_pic-link\\extras\\build_script_build\\build_script_build.build_script_build.295b7d9c-cgu.2.rcgu.o" "buck-out\\v2\\gen\\root\\fb50fd37ce946800\\third-party\\__semver-1.0.17-build-script-build__\\bin-pic-static_pic-link\\extras\\build_script_build\\build_script_build.build_script_build.295b7d9c-cgu.3.rcgu.o" "buck-out\\v2\\gen\\root\\fb50fd37ce946800\\third-party\\__semver-1.0.17-build-script-build__\\bin-pic-static_pic-link\\extras\\build_script_build\\build_script_build.build_script_build.295b7d9c-cgu.4.rcgu.o" "buck-out\\v2\\gen\\root\\fb50fd37ce946800\\third-party\\__semver-1.0.17-build-script-build__\\bin-pic-static_pic-link\\extras\\build_script_build\\build_script_build.build_script_build.295b7d9c-cgu.5.rcgu.o" "buck-out\\v2\\gen\\root\\fb50fd37ce946800\\third-party\\__semver-1.0.17-build-script-build__\\bin-pic-static_pic-link\\extras\\build_script_build\\build_script_build.build_script_build.295b7d9c-cgu.6.rcgu.o" "buck-out\\v2\\gen\\root\\fb50fd37ce946800\\third-party\\__semver-1.0.17-build-script-build__\\bin-pic-static_pic-link\\extras\\build_script_build\\build_script_build.build_script_build.295b7d9c-cgu.7.rcgu.o" "buck-out\\v2\\gen\\root\\fb50fd37ce946800\\third-party\\__semver-1.0.17-build-script-build__\\bin-pic-static_pic-link\\extras\\build_script_build\\build_script_build.build_script_build.295b7d9c-cgu.8.rcgu.o" "buck-out\\v2\\gen\\root\\fb50fd37ce946800\\third-party\\__semver-1.0.17-build-script-build__\\bin-pic-static_pic-link\\extras\\build_script_build\\build_script_build.build_script_build.295b7d9c-cgu.9.rcgu.o" "buck-out\\v2\\gen\\root\\fb50fd37ce946800\\third-party\\__semver-1.0.17-build-script-build__\\bin-pic-static_pic-link\\extras\\build_script_build\\build_script_build.32t6f3gzc5aq9zpj.rcgu.o" "/LIBPATH:C:\\Users\\steve\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib" "C:\\Users\\steve\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libstd-43ecf4bdbb5e1e31.rlib" "C:\\Users\\steve\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libpanic_unwind-03eee9fc7fe44114.rlib" "C:\\Users\\steve\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\librustc_demangle-f350829ea004292f.rlib" "C:\\Users\\steve\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libstd_detect-52475675c4d47720.rlib" "C:\\Users\\steve\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libhashbrown-d28662d15d021313.rlib" "C:\\Users\\steve\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libminiz_oxide-f5a9ba230b395602.rlib" "C:\\Users\\steve\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libadler-adcc534ee710b11e.rlib" "C:\\Users\\steve\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\librustc_std_workspace_alloc-75e91ab7163af4fe.rlib" "C:\\Users\\steve\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libunwind-6c926cb6aa5ed58c.rlib" "C:\\Users\\steve\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libcfg_if-b2a86741ec07ae31.rlib" "C:\\Users\\steve\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\liblibc-ecdbc3c93c11f60c.rlib" "C:\\Users\\steve\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\liballoc-39d88c9804b36d9f.rlib" "C:\\Users\\steve\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\librustc_std_workspace_core-2e112df156ffddae.rlib" "C:\\Users\\steve\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libcore-bd7106b7f2a04ee0.rlib" "C:\\Users\\steve\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libcompiler_builtins-d62a26418f17519a.rlib" "kernel32.lib" "advapi32.lib" "userenv.lib" "kernel32.lib" "ws2_32.lib" "bcrypt.lib" "msvcrt.lib" "legacy_stdio_definitions.lib" "/NXCOMPAT" "/LIBPATH:C:\\Users\\steve\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib" "/OUT:buck-out\\v2\\gen\\root\\fb50fd37ce946800\\third-party\\__semver-1.0.17-build-script-build__\\static_pic\\build_script_build.exe" "/OPT:REF,NOICF" "/DEBUG" "/NATVIS:C:\\Users\\steve\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\etc\\intrinsic.natvis" "/NATVIS:C:\\Users\\steve\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\etc\\liballoc.natvis" "/NATVIS:C:\\Users\\steve\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\etc\\libcore.natvis" "/NATVIS:C:\\Users\\steve\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\etc\\libstd.natvis" "@buck-out\\v2\\gen\\root\\fb50fd37ce946800\\third-party\\__semver-1.0.17-build-script-build__\\bin-pic-static_pic-link\\__build_script_build-link_linker_args.txt"
  = note: Microsoft (R) Incremental Linker Version 14.29.30148.0
          Copyright (C) Microsoft Corporation.  All rights reserved.

          "/NOLOGO"
          "C:\Users\steve\Documents\GitHub\buck-rust-hello\buck-out\v2\tmp\root\fb50fd37ce946800\third-party\__semver-1.0.17-build-script-build__\rustc\_buck_871788e481b8cd96\rustcqWXLE0\symbols.o"
          "buck-out\v2\gen\root\fb50fd37ce946800\third-party\__semver-1.0.17-build-script-build__\bin-pic-static_pic-link\extras\build_script_build\build_script_build.build_script_build.295b7d9c-cgu.0.rcgu.o"
          "buck-out\v2\gen\root\fb50fd37ce946800\third-party\__semver-1.0.17-build-script-build__\bin-pic-static_pic-link\extras\build_script_build\build_script_build.build_script_build.295b7d9c-cgu.1.rcgu.o"
          "buck-out\v2\gen\root\fb50fd37ce946800\third-party\__semver-1.0.17-build-script-build__\bin-pic-static_pic-link\extras\build_script_build\build_script_build.build_script_build.295b7d9c-cgu.10.rcgu.o"
          "buck-out\v2\gen\root\fb50fd37ce946800\third-party\__semver-1.0.17-build-script-build__\bin-pic-static_pic-link\extras\build_script_build\build_script_build.build_script_build.295b7d9c-cgu.11.rcgu.o"
          "buck-out\v2\gen\root\fb50fd37ce946800\third-party\__semver-1.0.17-build-script-build__\bin-pic-static_pic-link\extras\build_script_build\build_script_build.build_script_build.295b7d9c-cgu.12.rcgu.o"
          "buck-out\v2\gen\root\fb50fd37ce946800\third-party\__semver-1.0.17-build-script-build__\bin-pic-static_pic-link\extras\build_script_build\build_script_build.build_script_build.295b7d9c-cgu.13.rcgu.o"
          "buck-out\v2\gen\root\fb50fd37ce946800\third-party\__semver-1.0.17-build-script-build__\bin-pic-static_pic-link\extras\build_script_build\build_script_build.build_script_build.295b7d9c-cgu.14.rcgu.o"
          "buck-out\v2\gen\root\fb50fd37ce946800\third-party\__semver-1.0.17-build-script-build__\bin-pic-static_pic-link\extras\build_script_build\build_script_build.build_script_build.295b7d9c-cgu.15.rcgu.o"
          "buck-out\v2\gen\root\fb50fd37ce946800\third-party\__semver-1.0.17-build-script-build__\bin-pic-static_pic-link\extras\build_script_build\build_script_build.build_script_build.295b7d9c-cgu.2.rcgu.o"
          "buck-out\v2\gen\root\fb50fd37ce946800\third-party\__semver-1.0.17-build-script-build__\bin-pic-static_pic-link\extras\build_script_build\build_script_build.build_script_build.295b7d9c-cgu.3.rcgu.o"
          "buck-out\v2\gen\root\fb50fd37ce946800\third-party\__semver-1.0.17-build-script-build__\bin-pic-static_pic-link\extras\build_script_build\build_script_build.build_script_build.295b7d9c-cgu.4.rcgu.o"
          "buck-out\v2\gen\root\fb50fd37ce946800\third-party\__semver-1.0.17-build-script-build__\bin-pic-static_pic-link\extras\build_script_build\build_script_build.build_script_build.295b7d9c-cgu.5.rcgu.o"
          "buck-out\v2\gen\root\fb50fd37ce946800\third-party\__semver-1.0.17-build-script-build__\bin-pic-static_pic-link\extras\build_script_build\build_script_build.build_script_build.295b7d9c-cgu.6.rcgu.o"
          "buck-out\v2\gen\root\fb50fd37ce946800\third-party\__semver-1.0.17-build-script-build__\bin-pic-static_pic-link\extras\build_script_build\build_script_build.build_script_build.295b7d9c-cgu.7.rcgu.o"
          "buck-out\v2\gen\root\fb50fd37ce946800\third-party\__semver-1.0.17-build-script-build__\bin-pic-static_pic-link\extras\build_script_build\build_script_build.build_script_build.295b7d9c-cgu.8.rcgu.o"
          "buck-out\v2\gen\root\fb50fd37ce946800\third-party\__semver-1.0.17-build-script-build__\bin-pic-static_pic-link\extras\build_script_build\build_script_build.build_script_build.295b7d9c-cgu.9.rcgu.o"
          "buck-out\v2\gen\root\fb50fd37ce946800\third-party\__semver-1.0.17-build-script-build__\bin-pic-static_pic-link\extras\build_script_build\build_script_build.32t6f3gzc5aq9zpj.rcgu.o"
          "/LIBPATH:C:\Users\steve\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\x86_64-pc-windows-msvc\lib"
          "C:\Users\steve\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\x86_64-pc-windows-msvc\lib\libstd-43ecf4bdbb5e1e31.rlib"

          "C:\Users\steve\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\x86_64-pc-windows-msvc\lib\libpanic_unwind-03eee9fc7fe44114.rlib"
          "C:\Users\steve\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\x86_64-pc-windows-msvc\lib\librustc_demangle-f350829ea004292f.rlib"
          "C:\Users\steve\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\x86_64-pc-windows-msvc\lib\libstd_detect-52475675c4d47720.rlib"
          "C:\Users\steve\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\x86_64-pc-windows-msvc\lib\libhashbrown-d28662d15d021313.rlib"
          "C:\Users\steve\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\x86_64-pc-windows-msvc\lib\libminiz_oxide-f5a9ba230b395602.rlib"
          "C:\Users\steve\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\x86_64-pc-windows-msvc\lib\libadler-adcc534ee710b11e.rlib"
          "C:\Users\steve\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\x86_64-pc-windows-msvc\lib\librustc_std_workspace_alloc-75e91ab7163af4fe.rlib"
          "C:\Users\steve\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\x86_64-pc-windows-msvc\lib\libunwind-6c926cb6aa5ed58c.rlib"
          "C:\Users\steve\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\x86_64-pc-windows-msvc\lib\libcfg_if-b2a86741ec07ae31.rlib"
          "C:\Users\steve\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\x86_64-pc-windows-msvc\lib\liblibc-ecdbc3c93c11f60c.rlib"
          "C:\Users\steve\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\x86_64-pc-windows-msvc\lib\liballoc-39d88c9804b36d9f.rlib"
          "C:\Users\steve\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\x86_64-pc-windows-msvc\lib\librustc_std_workspace_core-2e112df156ffddae.rlib"
          "C:\Users\steve\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\x86_64-pc-windows-msvc\lib\libcore-bd7106b7f2a04ee0.rlib"
          "C:\Users\steve\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\x86_64-pc-windows-msvc\lib\libcompiler_builtins-d62a26418f17519a.rlib"
          "kernel32.lib"
          "advapi32.lib"
          "userenv.lib"
          "kernel32.lib"
          "ws2_32.lib"
          "bcrypt.lib"
          "msvcrt.lib"
          "legacy_stdio_definitions.lib"
          "/NXCOMPAT"
          "/LIBPATH:C:\Users\steve\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\x86_64-pc-windows-msvc\lib"
          "/OUT:buck-out\v2\gen\root\fb50fd37ce946800\third-party\__semver-1.0.17-build-script-build__\static_pic\build_script_build.exe"
          "/OPT:REF,NOICF"
          "/DEBUG"
          "/NATVIS:C:\Users\steve\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\etc\intrinsic.natvis"
          "/NATVIS:C:\Users\steve\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\etc\liballoc.natvis"
          "/NATVIS:C:\Users\steve\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\etc\libcore.natvis"
          "/NATVIS:C:\Users\steve\.rustup\toolchains\stable-x86_64-pc-windows-msvc\lib\rustlib\etc\libstd.natvis"
          "@buck-out\v2\gen\root\fb50fd37ce946800\third-party\__semver-1.0.17-build-script-build__\bin-pic-static_pic-link\__build_script_build-link_linker_args.txt"
          LINK : warning LNK4044: unrecognized option '/fuse-ld=lld'; ignored
          LINK : fatal error LNK1181: cannot open input file '@buck-out\v2\gen\root\fb50fd37ce946800\third-party\__semver-1.0.17-build-script-build__\bin-pic-static_pic-link\__build_script_build-link_linker_args.txt'

the image of telling link.exe "hey, /fuse-ld=lld" is pretty funny. That's obviously my fault for saying that the linker is link.exe for the system_cxx_toolchain, but the big failure here is not being able to open that linker args file. If I ls, the file is there, zero bytes in size. (Which I believe is expected, but I don't know why it cannot be opened)

If I don't do the link.exe hack, it is now finding clang++, but there are other problems:

C:\Users\steve\Documents\GitHub\buck-rust-hello〉buck2 run //src/bin:hello_world                                         05/02/2023 10:49:29 AM
File changed: toolchains//BUCK
Action failed: root//third-party:semver-1.0.17-build-script-build (rustc bin-pic-static_pic-link/build_script_build-link bin,pic,link [diag])
Local command returned non-zero exit code 1
Reproduce locally: `"buck-out\\v2\\gen\\prelude\\fb50fd37ce946800\\python_bootstrap\\tools\\__win_python_wrapper__\\win_ ...<omitted>... y\\__semver-1.0.17-build-script-build__\\bin-pic-static_pic-link\\build_script_build-link-diag.args" (run `buck2 log what-failed` to get the full command)`
stdout:
stderr:
error: linking with `buck-out\v2\gen\root\fb50fd37ce946800\third-party\__semver-1.0.17-build-script-build__\__linker_wrapper.bat` failed: exit code: 1
  |
  = note: "cmd" "/c" "buck-out\\v2\\gen\\root\\fb50fd37ce946800\\third-party\\__semver-1.0.17-build-script-build__\\__linker_wrapper.bat" "/NOLOGO" "C:\\Users\\steve\\Documents\\GitHub\\buck-rust-hello\\buck-out\\v2\\tmp\\root\\fb50fd37ce946800\\third-party\\__semver-1.0.17-build-script-build__\\rustc\\_buck_871788e481b8cd96\\rustc5QFfrE\\symbols.o" "buck-out\\v2\\gen\\root\\fb50fd37ce946800\\third-party\\__semver-1.0.17-build-script-build__\\bin-pic-static_pic-link\\extras\\build_script_build\\build_script_build.build_script_build.295b7d9c-cgu.0.rcgu.o" "buck-out\\v2\\gen\\root\\fb50fd37ce946800\\third-party\\__semver-1.0.17-build-script-build__\\bin-pic-static_pic-link\\extras\\build_script_build\\build_script_build.build_script_build.295b7d9c-cgu.1.rcgu.o" "buck-out\\v2\\gen\\root\\fb50fd37ce946800\\third-party\\__semver-1.0.17-build-script-build__\\bin-pic-static_pic-link\\extras\\build_script_build\\build_script_build.build_script_build.295b7d9c-cgu.10.rcgu.o" "buck-out\\v2\\gen\\root\\fb50fd37ce946800\\third-party\\__semver-1.0.17-build-script-build__\\bin-pic-static_pic-link\\extras\\build_script_build\\build_script_build.build_script_build.295b7d9c-cgu.11.rcgu.o" "buck-out\\v2\\gen\\root\\fb50fd37ce946800\\third-party\\__semver-1.0.17-build-script-build__\\bin-pic-static_pic-link\\extras\\build_script_build\\build_script_build.build_script_build.295b7d9c-cgu.12.rcgu.o" "buck-out\\v2\\gen\\root\\fb50fd37ce946800\\third-party\\__semver-1.0.17-build-script-build__\\bin-pic-static_pic-link\\extras\\build_script_build\\build_script_build.build_script_build.295b7d9c-cgu.13.rcgu.o" "buck-out\\v2\\gen\\root\\fb50fd37ce946800\\third-party\\__semver-1.0.17-build-script-build__\\bin-pic-static_pic-link\\extras\\build_script_build\\build_script_build.build_script_build.295b7d9c-cgu.14.rcgu.o" "buck-out\\v2\\gen\\root\\fb50fd37ce946800\\third-party\\__semver-1.0.17-build-script-build__\\bin-pic-static_pic-link\\extras\\build_script_build\\build_script_build.build_script_build.295b7d9c-cgu.15.rcgu.o" "buck-out\\v2\\gen\\root\\fb50fd37ce946800\\third-party\\__semver-1.0.17-build-script-build__\\bin-pic-static_pic-link\\extras\\build_script_build\\build_script_build.build_script_build.295b7d9c-cgu.2.rcgu.o" "buck-out\\v2\\gen\\root\\fb50fd37ce946800\\third-party\\__semver-1.0.17-build-script-build__\\bin-pic-static_pic-link\\extras\\build_script_build\\build_script_build.build_script_build.295b7d9c-cgu.3.rcgu.o" "buck-out\\v2\\gen\\root\\fb50fd37ce946800\\third-party\\__semver-1.0.17-build-script-build__\\bin-pic-static_pic-link\\extras\\build_script_build\\build_script_build.build_script_build.295b7d9c-cgu.4.rcgu.o" "buck-out\\v2\\gen\\root\\fb50fd37ce946800\\third-party\\__semver-1.0.17-build-script-build__\\bin-pic-static_pic-link\\extras\\build_script_build\\build_script_build.build_script_build.295b7d9c-cgu.5.rcgu.o" "buck-out\\v2\\gen\\root\\fb50fd37ce946800\\third-party\\__semver-1.0.17-build-script-build__\\bin-pic-static_pic-link\\extras\\build_script_build\\build_script_build.build_script_build.295b7d9c-cgu.6.rcgu.o" "buck-out\\v2\\gen\\root\\fb50fd37ce946800\\third-party\\__semver-1.0.17-build-script-build__\\bin-pic-static_pic-link\\extras\\build_script_build\\build_script_build.build_script_build.295b7d9c-cgu.7.rcgu.o" "buck-out\\v2\\gen\\root\\fb50fd37ce946800\\third-party\\__semver-1.0.17-build-script-build__\\bin-pic-static_pic-link\\extras\\build_script_build\\build_script_build.build_script_build.295b7d9c-cgu.8.rcgu.o" "buck-out\\v2\\gen\\root\\fb50fd37ce946800\\third-party\\__semver-1.0.17-build-script-build__\\bin-pic-static_pic-link\\extras\\build_script_build\\build_script_build.build_script_build.295b7d9c-cgu.9.rcgu.o" "buck-out\\v2\\gen\\root\\fb50fd37ce946800\\third-party\\__semver-1.0.17-build-script-build__\\bin-pic-static_pic-link\\extras\\build_script_build\\build_script_build.32t6f3gzc5aq9zpj.rcgu.o" "/LIBPATH:C:\\Users\\steve\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib" "C:\\Users\\steve\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libstd-43ecf4bdbb5e1e31.rlib" "C:\\Users\\steve\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libpanic_unwind-03eee9fc7fe44114.rlib" "C:\\Users\\steve\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\librustc_demangle-f350829ea004292f.rlib" "C:\\Users\\steve\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libstd_detect-52475675c4d47720.rlib" "C:\\Users\\steve\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libhashbrown-d28662d15d021313.rlib" "C:\\Users\\steve\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libminiz_oxide-f5a9ba230b395602.rlib" "C:\\Users\\steve\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libadler-adcc534ee710b11e.rlib" "C:\\Users\\steve\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\librustc_std_workspace_alloc-75e91ab7163af4fe.rlib" "C:\\Users\\steve\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libunwind-6c926cb6aa5ed58c.rlib" "C:\\Users\\steve\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libcfg_if-b2a86741ec07ae31.rlib" "C:\\Users\\steve\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\liblibc-ecdbc3c93c11f60c.rlib" "C:\\Users\\steve\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\liballoc-39d88c9804b36d9f.rlib" "C:\\Users\\steve\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\librustc_std_workspace_core-2e112df156ffddae.rlib" "C:\\Users\\steve\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libcore-bd7106b7f2a04ee0.rlib" "C:\\Users\\steve\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libcompiler_builtins-d62a26418f17519a.rlib" "kernel32.lib" "advapi32.lib" "userenv.lib" "kernel32.lib" "ws2_32.lib" "bcrypt.lib" "msvcrt.lib" "legacy_stdio_definitions.lib" "/NXCOMPAT" "/LIBPATH:C:\\Users\\steve\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib" "/OUT:buck-out\\v2\\gen\\root\\fb50fd37ce946800\\third-party\\__semver-1.0.17-build-script-build__\\static_pic\\build_script_build.exe" "/OPT:REF,NOICF" "/DEBUG" "/NATVIS:C:\\Users\\steve\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\etc\\intrinsic.natvis" "/NATVIS:C:\\Users\\steve\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\etc\\liballoc.natvis" "/NATVIS:C:\\Users\\steve\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\etc\\libcore.natvis" "/NATVIS:C:\\Users\\steve\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\etc\\libstd.natvis" "@buck-out\\v2\\gen\\root\\fb50fd37ce946800\\third-party\\__semver-1.0.17-build-script-build__\\bin-pic-static_pic-link\\__build_script_build-link_linker_args.txt"
  = note: clang++: error: no such file or directory: '/NOLOGO'
          clang++: error: no such file or directory: 'C:UserssteveDocumentsGitHubbuck-rust-hellobuck-outv2tmprootfb50fd37ce946800third-party__semver-1.0.17-build-script-build__rustc_buck_871788e481b8cd96rustc5QFfrEsymbols.o'
          clang++: error: no such file or directory: 'buck-outv2genrootfb50fd37ce946800third-party__semver-1.0.17-build-script-build__bin-pic-static_pic-linkextrasbuild_script_buildbuild_script_build.build_script_build.295b7d9c-cgu.0.rcgu.o'
          clang++: error: no such file or directory: 'buck-outv2genrootfb50fd37ce946800third-party__semver-1.0.17-build-script-build__bin-pic-static_pic-linkextrasbuild_script_buildbuild_script_build.build_script_build.295b7d9c-cgu.1.rcgu.o'
          clang++: error: no such file or directory: 'buck-outv2genrootfb50fd37ce946800third-party__semver-1.0.17-build-script-build__bin-pic-static_pic-linkextrasbuild_script_buildbuild_script_build.build_script_build.295b7d9c-cgu.10.rcgu.o'
          clang++: error: no such file or directory: 'buck-outv2genrootfb50fd37ce946800third-party__semver-1.0.17-build-script-build__bin-pic-static_pic-linkextrasbuild_script_buildbuild_script_build.build_script_build.295b7d9c-cgu.11.rcgu.o'
          clang++: error: no such file or directory: 'buck-outv2genrootfb50fd37ce946800third-party__semver-1.0.17-build-script-build__bin-pic-static_pic-linkextrasbuild_script_buildbuild_script_build.build_script_build.295b7d9c-cgu.12.rcgu.o'
          clang++: error: no such file or directory: 'buck-outv2genrootfb50fd37ce946800third-party__semver-1.0.17-build-script-build__bin-pic-static_pic-linkextrasbuild_script_buildbuild_script_build.build_script_build.295b7d9c-cgu.13.rcgu.o'
          clang++: error: no such file or directory: 'buck-outv2genrootfb50fd37ce946800third-party__semver-1.0.17-build-script-build__bin-pic-static_pic-linkextrasbuild_script_buildbuild_script_build.build_script_build.295b7d9c-cgu.14.rcgu.o'
          clang++: error: no such file or directory: 'buck-outv2genrootfb50fd37ce946800third-party__semver-1.0.17-build-script-build__bin-pic-static_pic-linkextrasbuild_script_buildbuild_script_build.build_script_build.295b7d9c-cgu.15.rcgu.o'
          clang++: error: no such file or directory: 'buck-outv2genrootfb50fd37ce946800third-party__semver-1.0.17-build-script-build__bin-pic-static_pic-linkextrasbuild_script_buildbuild_script_build.build_script_build.295b7d9c-cgu.2.rcgu.o'
          clang++: error: no such file or directory: 'buck-outv2genrootfb50fd37ce946800third-party__semver-1.0.17-build-script-build__bin-pic-static_pic-linkextrasbuild_script_buildbuild_script_build.build_script_build.295b7d9c-cgu.3.rcgu.o'
          clang++: error: no such file or directory: 'buck-outv2genrootfb50fd37ce946800third-party__semver-1.0.17-build-script-build__bin-pic-static_pic-linkextrasbuild_script_buildbuild_script_build.build_script_build.295b7d9c-cgu.4.rcgu.o'
          clang++: error: no such file or directory: 'buck-outv2genrootfb50fd37ce946800third-party__semver-1.0.17-build-script-build__bin-pic-static_pic-linkextrasbuild_script_buildbuild_script_build.build_script_build.295b7d9c-cgu.5.rcgu.o'
          clang++: error: no such file or directory: 'buck-outv2genrootfb50fd37ce946800third-party__semver-1.0.17-build-script-build__bin-pic-static_pic-linkextrasbuild_script_buildbuild_script_build.build_script_build.295b7d9c-cgu.6.rcgu.o'
          clang++: error: no such file or directory: 'buck-outv2genrootfb50fd37ce946800third-party__semver-1.0.17-build-script-build__bin-pic-static_pic-linkextrasbuild_script_buildbuild_script_build.build_script_build.295b7d9c-cgu.7.rcgu.o'
          clang++: error: no such file or directory: 'buck-outv2genrootfb50fd37ce946800third-party__semver-1.0.17-build-script-build__bin-pic-static_pic-linkextrasbuild_script_buildbuild_script_build.build_script_build.295b7d9c-cgu.8.rcgu.o'
          clang++: error: no such file or directory: 'buck-outv2genrootfb50fd37ce946800third-party__semver-1.0.17-build-script-build__bin-pic-static_pic-linkextrasbuild_script_buildbuild_script_build.build_script_build.295b7d9c-cgu.9.rcgu.o'
          clang++: error: no such file or directory: '/LIBPATH:C:Userssteve.rustuptoolchainsstable-x86_64-pc-windows-msvclibrustlibx86_64-pc-windows-msvclib'
          clang++: error: no such file or directory: 'C:Userssteve.rustuptoolchainsstable-x86_64-pc-windows-msvclibrustlibx86_64-pc-windows-msvcliblibstd-43ecf4bdbb5e1e31.rlib'
          clang++: error: no such file or directory: 'C:Userssteve.rustuptoolchainsstable-x86_64-pc-windows-msvclibrustlibx86_64-pc-windows-msvcliblibpanic_unwind-03eee9fc7fe44114.rlib'
          clang++: error: no such file or directory: 'C:Userssteve.rustuptoolchainsstable-x86_64-pc-windows-msvclibrustlibx86_64-pc-windows-msvcliblibrustc_demangle-f350829ea004292f.rlib'
          clang++: error: no such file or directory: 'C:Userssteve.rustuptoolchainsstable-x86_64-pc-windows-msvclibrustlibx86_64-pc-windows-msvcliblibstd_detect-52475675c4d47720.rlib'
          clang++: error: no such file or directory: 'C:Userssteve.rustuptoolchainsstable-x86_64-pc-windows-msvclibrustlibx86_64-pc-windows-msvcliblibminiz_oxide-f5a9ba230b395602.rlib'
          clang++: error: no such file or directory: 'C:Userssteve.rustuptoolchainsstable-x86_64-pc-windows-msvclibrustlibx86_64-pc-windows-msvcliblibadler-adcc534ee710b11e.rlib'
          clang++: error: no such file or directory: 'C:Userssteve.rustuptoolchainsstable-x86_64-pc-windows-msvclibrustlibx86_64-pc-windows-msvcliblibrustc_std_workspace_alloc-75e91ab7163af4fe.rlib'
          clang++: error: no such file or directory: 'C:Userssteve.rustuptoolchainsstable-x86_64-pc-windows-msvclibrustlibx86_64-pc-windows-msvcliblibunwind-6c926cb6aa5ed58c.rlib'
          clang++: error: no such file or directory: 'C:Userssteve.rustuptoolchainsstable-x86_64-pc-windows-msvclibrustlibx86_64-pc-windows-msvcliblibcfg_if-b2a86741ec07ae31.rlib'
          clang++: error: no such file or directory: 'C:Userssteve.rustuptoolchainsstable-x86_64-pc-windows-msvclibrustlibx86_64-pc-windows-msvclibliblibc-ecdbc3c93c11f60c.rlib'
          clang++: error: no such file or directory: 'C:Userssteve.rustuptoolchainsstable-x86_64-pc-windows-msvclibrustlibx86_64-pc-windows-msvclibliballoc-39d88c9804b36d9f.rlib'
          clang++: error: no such file or directory: 'C:Userssteve.rustuptoolchainsstable-x86_64-pc-windows-msvclibrustlibx86_64-pc-windows-msvcliblibrustc_std_workspace_core-2e112df156ffddae.rlib'
          clang++: error: no such file or directory: 'C:Userssteve.rustuptoolchainsstable-x86_64-pc-windows-msvclibrustlibx86_64-pc-windows-msvcliblibcore-bd7106b7f2a04ee0.rlib'
          clang++: error: no such file or directory: 'C:Userssteve.rustuptoolchainsstable-x86_64-pc-windows-msvclibrustlibx86_64-pc-windows-msvcliblibcompiler_builtins-d62a26418f17519a.rlib'
          clang++: error: no such file or directory: 'kernel32.lib'
          clang++: error: no such file or directory: 'advapi32.lib'
          clang++: error: no such file or directory: 'userenv.lib'
          clang++: error: no such file or directory: 'kernel32.lib'
          clang++: error: no such file or directory: 'ws2_32.lib'
          clang++: error: no such file or directory: 'bcrypt.lib'
          clang++: error: no such file or directory: 'msvcrt.lib'
          clang++: error: no such file or directory: 'legacy_stdio_definitions.lib'
          clang++: error: no such file or directory: '/NXCOMPAT'
          clang++: error: no such file or directory: '/LIBPATH:C:Userssteve.rustuptoolchainsstable-x86_64-pc-windows-msvclibrustlibx86_64-pc-windows-msvclib'
          clang++: error: no such file or directory: '/OUT:buck-outv2genrootfb50fd37ce946800third-party__semver-1.0.17-build-script-build__static_picbuild_script_build.exe'
          clang++: error: no such file or directory: '/OPT:REF,NOICF'
          clang++: error: no such file or directory: '/DEBUG'
          clang++: error: no such file or directory: '/NATVIS:C:Userssteve.rustuptoolchainsstable-x86_64-pc-windows-msvclibrustlibetcintrinsic.natvis'
          clang++: error: no such file or directory: '/NATVIS:C:Userssteve.rustuptoolchainsstable-x86_64-pc-windows-msvclibrustlibetcliballoc.natvis'
          clang++: error: no such file or directory: '/NATVIS:C:Userssteve.rustuptoolchainsstable-x86_64-pc-windows-msvclibrustlibetclibcore.natvis'
          clang++: error: no such file or directory: '/NATVIS:C:Userssteve.rustuptoolchainsstable-x86_64-pc-windows-msvclibrustlibetclibstd.natvis'
          clang++: error: no such file or directory: '@buck-outv2genrootfb50fd37ce946800third-party__semver-1.0.17-build-script-build__bin-pic-static_pic-link__build_script_build-link_linker_args.txt'
          clang++: error: no input files

It seems to be eating the slashes in the paths...

dtolnay commented 1 year ago

I made a repro of the slash eating issue that Steve is seeing. https://github.com/dtolnay/cxx/commit/5599af4b6e732ea03e10e33c5cbd2c4582c72f2b -- here is the failed build: https://github.com/dtolnay/cxx/actions/runs/4876821900/jobs/8700849033.

clang++: error: no such file or directory: 'D:acxxcxxbuck-outv2tmprootfb50fd37ce946800third-party__quote-1.0.26-build-script-build__rustc_buck_871788e481b8cd96rustcqh3M0esymbols.o'
...
clang++: error: no such file or directory: '/LIBPATH:C:Usersrunneradmin.rustuptoolchainsstable-x86_64-pc-windows-msvclibrustlibx86_64-pc-windows-msvclib'
clang++: error: no such file or directory: 'C:Usersrunneradmin.rustuptoolchainsstable-x86_64-pc-windows-msvclibrustlibx86_64-pc-windows-msvcliblibstd-43ecf4bdbb5e1e31.rlib'
...
clang++: error: no such file or directory: '@buck-outv2genrootfb50fd37ce946800third-party__quote-1.0.26-build-script-build__bin-pic-static_pic-link__build_script_build-link_linker_args.txt'
steveklabnik commented 1 year ago

With @dtolnay's latest work landing in the prelude, I can now build even non-trivial Rust projects with MSVC-flavored Rust. Woo!

lmvasquezg commented 1 year ago

Just tried it! Works like a charm. Thanks @dtolnay! With that our hello_world example works using this toolchain:

system_cxx_toolchain(
    name = "cxx_win",
    visibility = ["PUBLIC"],
    compiler = "windows",
    cxx_compiler = "cl",
    linker = "link.exe",
    compiler_type = "windows"
)
dtolnay commented 1 year ago

cxx_compiler = "cl"

Are cl.exe and ml64.exe typically available in path on Windows, or do we need to reimplement a way to discover them in some deeply nested Visual Studio directory?

The cc crate has 900 lines of code just dedicated to finding the location of the MSVC tools... https://github.com/rust-lang/cc-rs/blob/1.0.79/src/windows_registry.rs

I saw that in GitHub Actions, running https://github.com/egor-tensin/vs-shell is sufficient for making cl / ml64 do the right thing, but I am not sure whether this resembles a typical developer machine.

lmvasquezg commented 1 year ago

If you're using Visual Studios environment (which is usually the case for MSVC) they should be on the path. It could still be hard coded on the toolchain for when you have it checked-in like we do internally. Or just set up your command line like explained here

steveklabnik commented 1 year ago

If you're using Visual Studios environment (which is usually the case for MSVC)

image

Yeah, just to rephrase what you said, it's not on the path of shells by default, but if you explicitly select the Developer Command Prompt, you do get it. The norm in Rust isn't to use that prompt, and to fetch the tools directly (as you already posted.)

I can't speak to the norms of folks who do C++ development, though. So "usually" is a bit context dependent.