search

aspect-build / rules_swc

Bazel rules for the swc toolchain https://swc.rs/
https://docs.aspect.build/rules/aspect_rules_swc
Apache License 2.0
44 stars 24 forks source link

Failing when spawed remote #20

Closed alexeagle closed 2 years ago

alexeagle commented 2 years ago

Copied from Slack https://bazelbuild.slack.com/archives/CEZUUKQ6P/p1643627483904509

I'm trying out the SWC rules. Locally everything seems to work fine, however when enabling remote build execution I run into the following error message:

ERROR: /_work/frontend/frontend/packages/libs/icons/BUILD.bazel:7:11: Transpiling with swc //packages/libs/icons:tsconfig_transpile [swc packages/libs/icons/src/plain/Sort.tsx] failed: (Exit 34): Remote Execution Failure:

Unknown: Failed to run command: Failed to start process: fork/exec bazel-out/k8-opt-exec-EAC798B4/bin/external/aspect_rules_swc/swc/_cli_launcher.sh: no such file or directory

The action is executing the following command:

SWC_BINARY_PATH=external/swc_linux-x64-gnu/swc.linux-x64-gnu.node bazel-out/k8-opt-exec-EAC798B4-ST-0e4399772f5b/bin/external/aspect_rules_swc/swc/_cli_launcher.sh --no-swcrc packages/apps/self-billing-app/src/App.tsx --out-file bazel-out/k8-fastbuild-ST-4a519fd6d3e4/bin/packages/apps/self-billing-app/src/App.js -q

I looked into the input files of the action, specifically into the _cli_launcher.sh. This file is there, but it looks like it's a symlink to nothing?

> swc pwd
/Users/dhl/Downloads/352755f10846f9e2a50f6fc6a50dde30a7c7b04a6c0c31be3ddc49a6bcc89c64/bazel-out/k8-opt-exec-EAC798B4-ST-0e4399772f5b/bin/external/aspect_rules_swc/swc
> swc ls -l
total 0
lrw-------@  1 dhl  staff   100 Jan  1  1970 _cli_launcher.sh -> bazel-out/k8-opt-exec-EAC798B4-ST-0e4399772f5b/bin/external/aspect_rules_swc/swc/_cli_launcher.sh.ru
lrw-------@  1 dhl  staff   100 Jan  1  1970 _cli_launcher.sh.ru -> bazel-out/k8-opt-exec-EAC798B4-ST-0e4399772f5b/bin/external/aspect_rules_swc/swc/_cli_launcher.sh.ru
drwxr-xr-x@ 32 dhl  staff  1024 Jan  1  1970 _cli_launcher.sh.runfiles

I created a small repository to try out the SWC bazel rules on Buildbuddy: https://github.com/henkerik/swc-rbe-test

The builds fails on Buildbuddy with the same error message no such file or directory as on our own Buildbarn cluster.

alexeagle commented 2 years ago

FWIW this is where that _cli_launcher.sh gets returned to bazel along with some runfiles

https://github.com/aspect-build/rules_js/blob/3bb043cdef611277db0912c190e52e807d2c2b0c/js/private/nodejs_binary.bzl#L186-L197

We do have some path manipulation in that file to remove prefixes from paths, but I don't see anything that could explain the _cli_launcher.sh.ru truncation.

alexeagle commented 2 years ago

Also I still haven't managed to reproduce this, I don't have a buildbuddy API token and haven't taken the time to make my own RBE setup to test on.

bduffany commented 2 years ago

I was able to repro this and narrowed down the problem to the shebang line in _cli_launcher.sh, which turns out to be #!/usr/bin/bash when executed. At least with our default docker RBE image (based on marketplace.gcr.io/google/rbe-ubuntu16-04), /usr/bin/bash doesn't exist.

I see from that nodejs_binary.bzl file that the bash path is populated from the sh toolchain here: https://github.com/aspect-build/rules_js/blob/3bb043cdef611277db0912c190e52e807d2c2b0c/js/private/nodejs_binary.bzl#L129

So we probably need to update our RBE toolchain (https://github.com/buildbuddy-io/buildbuddy-toolchain) to configure an sh_toolchain, or maybe just provide a convenience symlink in our RBE image of /usr/bin/bash -> /bin/bash. (Curious why we run into this with SWC, and not other rules_nodejs rules, though)

alexeagle commented 2 years ago

We are in a migration, rules swc is the first that already uses our fork at https://github.com/aspect-build/rules_js/blob/main/js/private/js_binary.bzl The README of that repo explains the motivation. So that's probably why it's different behavior from rules using the nodejs_binary rule's bash launcher. Thanks for tracking this down! Please let me know if the rule ought to do something different like '/usr/bin/env bash'

bduffany commented 2 years ago

It almost seems like the default sh_toolchain should set /usr/bin/env bash as the bash "path," and then this problem would go away. But I guess that might break some things because it's not just a path, but rather a path + arg.

I guess I would lean towards using #!/usr/bin/env bash as the shebang because it seems to "just work" and it's what rules_nodejs already uses (I think?). It would also reduce friction when getting started with RBE which is already a pain due to platforms + toolchains.

But it might also be a good thing to bias towards flexibility and make the shell explicitly configurable via a toolchain. There may be a way to get the best of both worlds somehow, though I'm not familiar enough with toolchains to say :shrug:

alexeagle commented 2 years ago

Pasting what you wrote in Slack:

In the meantime you can add this to your top-level BUILD file and then run your RBE build with --extra_toolchains=//:sh_toolchain

load("@bazel_tools//tools/sh:sh_toolchain.bzl", "sh_toolchain")
sh_toolchain(
    name = "sh",
    path = "/bin/bash",
)
toolchain(
    name = "sh_toolchain",
    toolchain = ":sh",
    toolchain_type = "@bazel_tools//tools/sh:toolchain_type",
)
alexeagle commented 2 years ago

I think it's fixed now by https://github.com/aspect-build/rules_js/pull/57 which changed the launcher template to #!/usr/bin/env bash. Though it does feel like all RBE systems ought to provide a bash toolchain?