Using standalone binary on Alpine

[errordeveloper]

Since #1492 we have good news is – one can just download the standalone binary and no need to install Java, however it this binary still depends on glibc...

With libc6-compat installed, I'm seeing this:

dfc0b8b9e9f8:/go# ldd /usr/local/bin/bazel
        /lib64/ld-linux-x86-64.so.2 (0x7f2384c48000)
        librt.so.1 => /lib64/ld-linux-x86-64.so.2 (0x7f2384c48000)
        libdl.so.2 => /lib64/ld-linux-x86-64.so.2 (0x7f2384c48000)
        libpthread.so.0 => /lib64/ld-linux-x86-64.so.2 (0x7f2384c48000)
        libm.so.6 => /lib64/ld-linux-x86-64.so.2 (0x7f2384c48000)
        libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x7f23848f6000)
        libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x7f23846e4000)
        libc.so.6 => /lib64/ld-linux-x86-64.so.2 (0x7f2384c48000)
Error relocating /usr/local/bin/bazel: __realpath_chk: symbol not found
Error relocating /usr/local/bin/bazel: __memcpy_chk: symbol not found
Error relocating /usr/local/bin/bazel: __sprintf_chk: symbol not found
dfc0b8b9e9f8:/go#

Here is an edited version if what I have:

FROM golang:1.10-alpine AS build

RUN apk add --update \
      curl \
      git \
      make \
      py-pip \
      python \
      python-dev \
      && true

RUN mkdir /out
RUN mkdir -p /out/etc/apk && cp -r /etc/apk/* /out/etc/apk/
RUN apk add --no-cache --initdb --root /out \
    alpine-baselayout \
    busybox \
    ca-certificates \
    coreutils \
    git \
    libc6-compat \
    libgcc \
    libstdc++ \
    python \
    && true

## <snip> build Skaffold, which uses bazel

WORKDIR /out

RUN ln -s /lib /lib64

ENV BAZEL_VERSION 0.16.1
RUN curl --silent --location "https://github.com/bazelbuild/bazel/releases/download/${BAZEL_VERSION}/bazel-${BAZEL_VERSION}-linux-x86_64" --output usr/local/bin/bazel \
    && chmod +x usr/local/bin/bazel

## <snip> install other runtime dependencies of Skaffold

FROM scratch
CMD skaffold
COPY --from=build  /out /

I wonder if bazel actually has to use glibc or not.

[errordeveloper]

related https://github.com/bazelbuild/bazel/issues/5909

[tyhal]

Do you need glibc to only run bazel or will it also use it for builds?

As in, does the bazel build produced binaries have musl or glibc in them?

[errordeveloper]

@tylerhale good question! I would be surprised if Bazel intervened at that level. I believe it hands off to C compiler, and it should be up to the user to configure C compiler to use whatever C standard library their application requirers, and in most cases user would not switch away from the system C library (which is glibc on most Linux systems, and something else on other ones, e.g. musl or bionic)

[philwo]

@errordeveloper What would you like Bazel to do to improve the situation?

Our official "Linux" release binaries are built on Ubuntu 14.04 LTS. I understand that they won't work on more minimal distros without a glibc. I think a good way to solve this is to bootstrap Bazel for your platform, but I understand that #5909 is blocking that.

Would it be enough to solve #5909 or is there anything we should do on this issue, too?

[hnnsgstfssn]

@philwo Could the official release binaries perhaps include a statically linked binary as part of of this issue or should that be a separate request? I would find a statically linked official release binary helpful and preferable. Linking statically, if I understand correctly, fixes this issue and would also cover all distributions regardless of C library implementation.

[mihaigalos]

Reproducible on alpine:3.11.3.

[stabai]

Any update on this? Bazel is the only part of our dev toolkit that we can't get to run on Alpine, and it makes things very difficult. Either having an up-to-date APK package to install Bazel/Bazelisk (ideal), a set of instructions for getting Bazel to work on Alpine, or a prebuilt image that can be inherited from would be extremely useful.

[yesudeep]

I'm new to docker, but I've got a RBE container image working with this Dockerfile (remove what you don't use/need). It builds bazel from source using a release distribution and plants it at /usr/local/bin/bazel. It currently doesn't verify sha256sums or .sig, but I'm working on adding that.

FROM alpine:edge

RUN \
    sed -i -e 's/^http:/#http:/' -e 's/^#http:\(.*edge.*\)/https:\1/' /etc/apk/repositories \
    && apk update \
    && apk add --no-cache \
    autoconf \
    autoconf-archive \
    automake \
    bash \
    bash-completion \
    binutils-gold \
    clang \
    cmake \
    coreutils \
    curl \
    g++ \
    gcc \
    gcompat \
    gettext \
    git \
    gmp-dev \
    go \
    gpg \
    libc-dev \
    libc6-compat \
    libffi-dev \
    libtool \
    linux-headers \
    m4 \
    make \
    musl-dev \
    ncurses-dev \
    openjdk11-jdk \
    openjdk17-jdk \
    patch \
    perl \
    pv \
    py3-pip \
    python3-dev \
    shadow \
    tar \
    texinfo \
    tig \
    tree \
    unzip \
    vim \
    wget \
    xz \
    zip

RUN \
    mkdir -p /tmp/bazel-release \
    && chmod a+rwx /tmp/bazel-release \
    && cd /tmp/bazel-release \
    && wget -O bazel-release-dist.zip \
       https://github.com/bazelbuild/bazel/releases/download/6.0.0/bazel-6.0.0-dist.zip \
    && unzip bazel-release-dist.zip \
    && env JAVA_HOME=/usr/lib/jvm/java-11-openjdk \
       EXTRA_BAZEL_ARGS="--tool_java_runtime_version=local_jdk" \
       bash ./compile.sh \
    && cp output/bazel /usr/local/bin/ \
    && cd

RUN printf 'export JAVA_HOME=$(dirname $(dirname $(readlink -f $(type -P javac))))' > /etc/profile.d/java_home.sh

RUN printf 'startup --output_user_root=/tmp/bazel\n\
run --color=yes\n\
build --color=yes\n\
build --verbose_failures\n\
build --enable_runfiles\n\
test --test_output=errors\n\
test --test_verbose_timeout_warnings\n\
test --test_summary=terse\n'\
> /etc/bazel.bazelrc

ENV PATH="/usr/local/bin:$PATH"

WORKDIR /src

[Mizux]

My 2cents, even with libc6-compat bazel is still unusable...

here a trace using bazelisk on a Alpine:edge Dockerfile

FROM alpine:edge AS env

# Install system build dependencies
ENV PATH=/usr/local/bin:$PATH
RUN apk add --no-cache git build-base linux-headers zlib-dev
#RUN apk add --no-cache -X http://dl-cdn.alpinelinux.org/alpine/edge/testing bazel6

# Install Bazelisk
RUN wget \
https://github.com/bazelbuild/bazelisk/releases/download/v1.19.0/bazelisk-linux-amd64 \
&& chmod +x bazelisk-linux-amd64 \
&& mv bazelisk-linux-amd64 /usr/local/bin/bazel

# Install OpenJDK17
# note: default-jvm will now point to java-17-openjdk
RUN apk add --no-cache openjdk17 --repository=http://dl-cdn.alpinelinux.org/alpine/edge/testing/
# Remove infinite loop since jre point to the current directory
# otherwise bazel issue an error and stop...
RUN rm /usr/lib/jvm/default-jvm/jre

ENV JAVA_HOME=/usr/lib/jvm/default-jvm
ENV PATH=$JAVA_HOME/bin:$PATH
...

Here the relevant trace...

/home/project # bazel --version
2023/12/22 13:33:08 could not run Bazel: could not start Bazel: fork/exec /root/.cache/bazelisk/downloads/sha256/8b24f70542f9736e0ccbf9cb3335cb9f049b5a6787f1e5997ed1da80b9bbd46e/bin/bazel: no such file or directory

/home/project # apk add libc6-compat
fetch https://dl-cdn.alpinelinux.org/alpine/edge/main/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/edge/community/x86_64/APKINDEX.tar.gz
(1/3) Installing musl-obstack (1.2.3-r2)
(2/3) Installing libucontext (1.2-r2)
(3/3) Installing gcompat (1.1.0-r4)
OK: 651 MiB in 104 packages

/home/project # bazel --version
bazel 7.0.0

/home/project # bazel build -c opt ...
Extracting Bazel installation...
Starting local Bazel server and connecting to it...
Server crashed during startup. Now printing /root/.cache/bazel/_bazel_root/238276933ebafbe5412dd515892860d4/server/jvm.out
Error occurred during initialization of VM
Unable to load jimage library: /root/.cache/bazel/_bazel_root/install/89a68939cbf63eb54205fdf943a58b15/embedded_tools/jdk/lib/libjimage.so

[avdv]

/home/project # bazel --version
bazel 7.0.0

/home/project # bazel build -c opt ...
Extracting Bazel installation...
Starting local Bazel server and connecting to it...
Server crashed during startup. Now printing /root/.cache/bazel/_bazel_root/238276933ebafbe5412dd515892860d4/server/jvm.out
Error occurred during initialization of VM
Unable to load jimage library: /root/.cache/bazel/_bazel_root/install/89a68939cbf63eb54205fdf943a58b15/embedded_tools/jdk/lib/libjimage.so

This seems to use the embedded JDK. Have you tried to use the locally installed JDK by using the --server_javabase flag, passing in the Java home directory?

[x-yuri]

It seems like these days it fails for a different reason:

$ docker run --rm -it alpine:3.20
/ # apk add alpine-sdk doas
/ # echo 'permit nopass root' >> /etc/doas.conf
/ # abuild-keygen -ai
/ # git clone https://gitlab.alpinelinux.org/alpine/aports
/ # cd aports/testing/bazel5
/aports/testing/bazel5 # abuild -rF
...
ERROR: /tmp/bazel_XXOBndJN/out/external/com_google_absl/absl/base/BUILD.bazel:151:11: Compiling absl/base/internal/low_level_alloc.cc [for host] failed: (Exit 1): gcc failed: error executing command
  (cd /tmp/bazel_XXOBndJN/out/execroot/io_bazel && \
  exec env - \
    PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin \
    PWD=/proc/self/cwd \
  /usr/bin/gcc -U_FORTIFY_SOURCE -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer -g0 -O2 '-D_FORTIFY_SOURCE=1' -DNDEBUG -ffunction-sections -fdata-sections '-std=c++0x' -MD -MF bazel-out/host/bin/external/com_google_absl/absl/base/_objs/malloc_internal/low_level_alloc.d '-frandom-seed=bazel-out/host/bin/external/com_google_absl/absl/base/_objs/malloc_internal/low_level_alloc.o' -iquote external/com_google_absl -iquote bazel-out/host/bin/external/com_google_absl -g0 -g0 -Wall -Wextra -Wcast-qual -Wconversion-null -Wformat-security -Wmissing-declarations -Woverlength-strings -Wpointer-arith -Wundef -Wunused-local-typedefs -Wunused-result -Wvarargs -Wvla -Wwrite-strings -DNOMINMAX -fno-canonical-system-headers -Wno-builtin-macro-redefined '-D__DATE__="redacted"' '-D__TIMESTAMP__="redacted"' '-D__TIME__="redacted"' -c external/com_google_absl/absl/base/internal/low_level_alloc.cc -o bazel-out/host/bin/external/com_google_absl/absl/base/_objs/malloc_internal/low_level_alloc.o)
# Configuration: 824d3ab37baf419e66d9c888526b45a58e80123e4abb3db110a449d73e522710
# Execution platform: //:default_host_platform
In file included from external/com_google_absl/absl/base/internal/low_level_alloc.cc:26:
external/com_google_absl/absl/base/internal/direct_mmap.h:75:25: error: 'off64_t' has not been declared
   75 |                         off64_t offset) noexcept {
      |                         ^~~~~~~
Target //src:bazel_nojdk failed to build
INFO: Elapsed time: 431.640s, Critical Path: 33.49s
INFO: 407 processes: 27 internal, 380 local.
FAILED: Build did NOT complete successfully

ERROR: Could not build Bazel
>>> ERROR: bazel5: build failed
...

[Mizux]

• https://github.com/abseil/abseil-cpp/issues/1473 ?