pybind / pybind11_bazel

Bazel wrapper around the pybind11 repository
Other
100 stars 54 forks source link

Cannot use these rules with RBEs #62

Closed 8W9aG closed 7 months ago

8W9aG commented 10 months ago

When trying to use these rules with an environment containing remote build executions, I get the following issue:

ERROR: /private/var/tmp/XXX/8b6b555df13b04c162b6251aa1bfe8e3/external/local_config_python/BUILD:67:8: Executing genrule @local_config_python//:python_include failed: Genrule spawn cannot be executed with any of the available strategies: [remote]. Your --spawn_strategy, --genrule_strategy and/or --strategy flags are probably too strict. Visit https://github.com/bazelbuild/bazel/issues/7480 for advice
Target //third_party/man-group/ArcticDB:ArcticDB failed to build
ERROR: /private/var/tmp/XXX/8b6b555df13b04c162b6251aa1bfe8e3/external/local_config_python/BUILD:45:11 Middleman _middlemen/@local_Uconfig_Upython_S_S_Cpython_Uheaders-cc_library-compile failed: Genrule spawn cannot be executed with any of the available strategies: [remote]. Your --spawn_strategy, --genrule_strategy and/or --strategy flags are probably too strict. Visit https://github.com/bazelbuild/bazel/issues/7480 for advice

This is when using the following in WORKSPACE:

http_archive(
    name = "rules_python",
    url = "https://github.com/bazelbuild/rules_python/archive/refs/tags/0.25.0.zip",
    sha256 = "9ffcbf19b197153d7ceafb126eb05b7b5ce847aa4e1745f03feed298b4940e2c",
    strip_prefix = "rules_python-0.25.0",
)

load("@rules_python//python:repositories.bzl", "python_register_toolchains")

python_register_toolchains(
    name = "python3_9",
    python_version = "3.9",
)

load("@python3_9//:defs.bzl", "interpreter")

python_configure(
    name = "local_config_python",
    python_interpreter_target = interpreter,
)

It seems that pybind11_bazel forces things to run locally due to local = 1 in the following piece of code: https://github.com/pybind/pybind11_bazel/blob/c65db0ac44ff3cd790a803e4a804e2cc806cf641/python_configure.bzl#L101

I was wondering if there was a reason local was forced here, and if not can I either remove it or add an option to remove this flag?

This commit seems to add the local option: https://github.com/pybind/pybind11_bazel/commit/992381ced716ae12122360b0fbadbc3dda436dbf

Having said that local is always true, so maybe it just needs to be propagated?

rwgk commented 10 months ago

@chuckx do you have advice?

From my side, with my very limited bazel knowledge: Currently we have a pretty low bar: if you have changes that make pybind11_bazel work for your purposes, I could run some tests, if nothing is broken it'll look good to me.

Optional: If you could add GHA tests for the things you care about, that would be ideal. — Currently we don't have any testing here at all, but we're testing indirectly via pybind11_abseil and pybind11_protobuf.

8W9aG commented 10 months ago

Made a PR here: https://github.com/pybind/pybind11_bazel/pull/63

junyer commented 7 months ago

As of the v2.11.1.bzl.2 release, pybind11_bazel uses rules_python. I think everything should work far more smoothly with RBE.