facebook / buck2

Build system, successor to Buck
https://buck2.build/
Apache License 2.0
3.58k stars 222 forks source link

symbolic links for resources used in sh_binary rule not working for windows remote execution. #674

Open valadez-abel opened 5 months ago

valadez-abel commented 5 months ago

Even after bypassing #673, symbolic links are not properly getting propagated to the remote execution executors for windows:

Action failed: toolchains//:test_re (test)
Remote command returned non-zero exit code 1
stdout:

C:\worker\work\4\1\exec>setlocal EnableDelayedExpansion 

C:\worker\work\4\1\exec>set __RESOURCES_ROOT=.\resources 

C:\worker\work\4\1\exec>set __SRC=C:\worker\work\4\1\exec\buck-out\v2\gen\prelude\749e8add6e5f1364\python_bootstrap\tools\__win_python_wrapper__\win_python_wrapper.bat 

C:\worker\work\4\1\exec>for %a in ("C:\worker\work\4\1\exec\buck-out\v2\gen\prelude\749e8add6e5f1364\python_bootstrap\tools\__win_python_wrapper__\win_python_wrapper.bat") do set "__SCRIPT_DIR=%~dpa" 

C:\worker\work\4\1\exec>set "__SCRIPT_DIR=C:\worker\work\4\1\exec\buck-out\v2\gen\prelude\749e8add6e5f1364\python_bootstrap\tools\__win_python_wrapper__\" 

C:\worker\work\4\1\exec>set BUCK_SH_BINARY_VERSION_UNSTABLE=2 

C:\worker\work\4\1\exec>set BUCK_PROJECT_ROOT=C:\worker\work\4\1\exec\buck-out\v2\gen\prelude\749e8add6e5f1364\python_bootstrap\tools\__win_python_wrapper__\\.\resources 

C:\worker\work\4\1\exec>set BUCK_DEFAULT_RUNTIME_RESOURCES=C:\worker\work\4\1\exec\buck-out\v2\gen\prelude\749e8add6e5f1364\python_bootstrap\tools\__win_python_wrapper__\\.\resources 

C:\worker\work\4\1\exec>C:\worker\work\4\1\exec\buck-out\v2\gen\prelude\749e8add6e5f1364\python_bootstrap\tools\__win_python_wrapper__\\.\resources\win_python_wrapper.bat buck-out\v2\gen\prelude\749e8add6e5f1364\cxx\tools\__dep_file_processor__\__dep_file_processor__ buck-out\v2\gen-anon\toolchains\749e8add6e5f1364443bc5d1e95a38d6\__rbox-python__\out\rbox-python\python.exe buck-out\v2\gen\prelude\749e8add6e5f1364\cxx\tools\__dep_file_processor__\dep_file_processor.py 
stderr:
**The system cannot find the path specified.**

Changing: https://github.com/facebook/buck2/blob/2214ec241cefe357faf499e30b26629ce6728c8c/prelude/sh_binary.bzl#L38 to use copied_dirs bypasses the problem.

Happy to provide a fix for this, as well as #673

JakobDegen commented 4 months ago

@valadez-abel do you have a self-contained repro for this issue? I'm asking because internally at Meta we're using these rules with RE quite successfully, and so I'd like to understand what the difference is

valadez-abel commented 4 months ago

I don't currently have a self contained repro for this. I can try to create one using zig (won't be able to package vs / windows sdk), as I'm hitting this as soon as the prelude / shell scripts to run the python kicks in, so any cpp file should do the trick. Give me a few days, though I highly suspect that there's a combination in play due to how Engflow's RE is setup vs how Meta's is setup.