emscripten-core / emsdk

Emscripten SDK
http://emscripten.org
Other
2.92k stars 660 forks source link

Bazel: Python3 not found #1376

Closed allsey87 closed 2 months ago

allsey87 commented 2 months ago

I have a very minimal setup, trying to compile a hello world C++ example to WebAssembly. Have followed the instructions in this repo but I keep running into the same error: /usr/bin/env: 'python3': No such file or directory

bazel build //main-emscripten:hello-world-wasm --sandbox_debug
INFO: Analyzed target //main-emscripten:hello-world-wasm (84 packages loaded, 6188 targets configured).
ERROR: /workspaces/builder-bazel/main-emscripten/BUILD:11:15: Action main-emscripten/hello-world-wasm/hello-world.js failed: (Exit 127): process-wrapper failed: error executing Action command 
  (cd /home/developer/.cache/bazel/_bazel_developer/25e07d78077dfe1eca932359d50e41ef/sandbox/processwrapper-sandbox/6/execroot/_main && \
  exec env - \
    TMPDIR=/tmp \
  /home/developer/.cache/bazel/_bazel_developer/install/c8ddc30f81d03fbb6c764cadbda081c8/process-wrapper '--timeout=0' '--kill_delay=15' '--stats=/home/developer/.cache/bazel/_bazel_developer/25e07d78077dfe1eca932359d50e41ef/sandbox/processwrapper-sandbox/6/stats.out' bazel-out/k8-opt-exec-ST-13d3ddad9198/bin/external/emsdk/emscripten_toolchain/wasm_binary --allow_empty_outputs --archive bazel-out/wasm-fastbuild-ST-de55c182e546/bin/main-emscripten/hello-world --outputs bazel-out/k8-fastbuild/bin/main-emscripten/hello-world-wasm/hello-world.js,bazel-out/k8-fastbuild/bin/main-emscripten/hello-world-wasm/hello-world.wasm,bazel-out/k8-fastbuild/bin/main-emscripten/hello-world-wasm/hello-world.wasm.map,bazel-out/k8-fastbuild/bin/main-emscripten/hello-world-wasm/hello-world.js.mem,bazel-out/k8-fastbuild/bin/main-emscripten/hello-world-wasm/hello-world.fetch.js,bazel-out/k8-fastbuild/bin/main-emscripten/hello-world-wasm/hello-world.worker.js,bazel-out/k8-fastbuild/bin/main-emscripten/hello-world-wasm/hello-world.data,bazel-out/k8-fastbuild/bin/main-emscripten/hello-world-wasm/hello-world.js.symbols,bazel-out/k8-fastbuild/bin/main-emscripten/hello-world-wasm/hello-world.wasm.debug.wasm,bazel-out/k8-fastbuild/bin/main-emscripten/hello-world-wasm/hello-world.html)
/usr/bin/env: 'python3': No such file or directory
Target //main-emscripten:hello-world-wasm failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 0.833s, Critical Path: 0.07s
INFO: 2 processes: 2 internal.
ERROR: Build did NOT complete successfully

Python3 is available on my system, but as I understand the Bazel system is designed to use a separate interpreter by default for better reproducibility. I tried to fix this by installing python_rules using bzlmod following these instructions for the dev-only use case, however, I get the same error.

How should I configure Python3/Bazel so that Emscripten has what it needs to do the build?

allsey87 commented 2 months ago

I tried to place the python_rules stuff into WORKSPACE instead of MODULE.bazel, this is currently what my WORKSPACE file looks like:

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

http_archive(
    name = "rules_python",
    sha256 = "c68bdc4fbec25de5b5493b8819cfc877c4ea299c0dcb15c244c5a00208cde311",
    strip_prefix = "rules_python-0.31.0",
    url = "https://github.com/bazelbuild/rules_python/releases/download/0.31.0/rules_python-0.31.0.tar.gz",
)

load("@rules_python//python:repositories.bzl", "py_repositories")

py_repositories()

load("@rules_python//python:repositories.bzl", "python_register_toolchains")

python_register_toolchains(
    name = "python_3_11_3",
    python_version = "3.11.3",
)

load("@python_3_11_3//:defs.bzl", "interpreter")

http_archive(
    name = "emsdk",
    sha256 = "8b5b3433eb732dcc7643a2707a12fd5cbe793a5dadbbae9a60c24a737a78fe33",
    strip_prefix = "emsdk-3.1.45/bazel",
    url = "https://github.com/emscripten-core/emsdk/archive/refs/tags/3.1.45.tar.gz",
)

load("@emsdk//:deps.bzl", emsdk_deps = "deps")
emsdk_deps()

load("@emsdk//:emscripten_deps.bzl", emsdk_emscripten_deps = "emscripten_deps")
emsdk_emscripten_deps(emscripten_version = "3.1.45")

load("@emsdk//:toolchains.bzl", "register_emscripten_toolchains")
register_emscripten_toolchains()

This doesn't change anything though, I still get the same error about python3 not being found... This is with Bazel 7.1.1 .

allsey87 commented 2 months ago

This error also happens when cloning emsdk and attempting to build the hello-world-wasm target with bazel build :hello-world-wasm.

For reference, I am building inside of a Docker container:

FROM python:3.11.6-slim-bookworm
ENV SHELL /bin/bash
ARG user_id=1000
ARG group_id=1000
ARG bazel_version=7.1.1

RUN apt-get update && apt-get install --assume-yes --no-install-recommends \
    build-essential \
    ca-certificates \
    curl \
    git \
    openjdk-17-jdk \
    sudo \
    unzip \
    zip \
    && rm -rf /var/lib/apt/lists/*

# Download Bazel
WORKDIR /tmp
RUN curl \
    --fail \
    --location \
    --remote-name \
    "https://github.com/bazelbuild/bazel/releases/download/${bazel_version}/bazel-${bazel_version}-linux-x86_64"
RUN mv "bazel-${bazel_version}-linux-x86_64" /usr/local/bin/bazel
RUN chmod +x /usr/local/bin/bazel
# Download Bazel buildtools
RUN curl \
    --fail \
    --location \
    --remote-name \
    "https://github.com/bazelbuild/buildtools/releases/download/v${bazel_version}/buildifier-linux-amd64"
RUN mv "buildifier-linux-amd64" /usr/local/bin/buildifier
RUN chmod +x /usr/local/bin/buildifier

# Create a developer user
ARG user_id=1000
ARG group_id=1000
RUN groupadd --gid ${group_id} developer && \
    useradd --uid ${user_id} --gid ${group_id} -m developer -s /bin/bash && \
    echo developer ALL=\(root\) NOPASSWD:ALL > /etc/sudoers.d/developer && \
    chmod 0440 /etc/sudoers.d/developer

# Switch the developer user
USER developer
allsey87 commented 2 months ago

After a lot of trial and error, I managed to solve this by changing the Docker image's base layer from python:3.11.6-slim-bookworm to debian:bookworm-slim and installing python3 with apt.

As I understand, the issue was that python-rules needs a local interpreter to bootstrap the interpreter that is going to be used by python-rules. However, due to however python-rules/bazel is handling PATH etc, the difference in how Python was installed was enough to break stuff.