search

bazelbuild / bazel

a fast, scalable, multi-language and extensible build system
https://bazel.build
Apache License 2.0
23.05k stars 4.03k forks source link

linux-sandbox-pid1.cc is not compatible with setting TMPDIR #23217

Open noodle-rubrik opened 1 month ago

noodle-rubrik commented 1 month ago

Description of the bug:

Basic format of the error is: src/main/tools/linux-sandbox-pid1.cc:320: "mount(/home/ubuntu/sdmain/logs, /home/ubuntu/sdmain/logs, nullptr, MS_BIND | MS_REC, nullptr)": Permission denied

This is occurring as root user, and the /home/ubuntu/sdmain/logs is what we've set the $TMPDIR env var to.

According to https://github.com/bazelbuild/bazel/issues/5900, this is an old bug from 2018 in which setting TMPDIR is not compatible with linux-sandbox.cc. (I'm able to comment on the issue, but I can't see how to re-open it after the bot closed it.) https://github.com/bazelbuild/bazel/issues/5900#issuecomment-2257129539

For our tests, we've edited the default spawn strategy in our namespace to replace the generic sandbox with processwrapper-sandbox, but it took us some time to find the above issue. If the underlying incompatibility can't be fixed easily, it would be nice if spawn strategy could see that TMPDIR is set, and not attempt to use the linux-sandbox in that case.

Which category does this issue belong to?

No response

What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

Not sure what triggers linux-sandbox in general. In our environment, we've been using TMPDIR for many years, and recently upgraded to Bazel 7 and Ubuntu 20 (in that order), with the latter triggering this bug.

Which operating system are you running Bazel on?

Ubuntu 22.04.4 LTS

What is the output of bazel info release?

release 7.1.0

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

No response

What's the output of git remote get-url origin; git rev-parse HEAD ?

No response

If this is a regression, please try to identify the Bazel commit where the bug was introduced with bazelisk --bisect.

No response

Have you found anything relevant by searching the web?

https://github.com/bazelbuild/bazel/issues/5900#issuecomment-2257129539

Any other information, logs, or outputs that you want to share?

No response

meisterT commented 1 month ago

cc @oquenchil

Yasumoto commented 1 month ago

I'm also experiencing this on 7.2.1 on the Pop_OS! 24.04 alpha (which is based on Ubuntu 24.04). In the interim, unsetting TMPDIR unblocked me.

oquenchil commented 1 week ago

Can you provide more detailed instructions for reproducing this? I'd need the exact command line.

I have been trying to reproduce with:

TMPDIR=/home/oquenchil/tmpdir USE_BAZEL_VERSION=7.1.0 bazelisk build --action_env=TMPDIR=/home/oquenchil/tmpdir --sandbox_debug :foo

but I don't see any errorrs. I tried with and without root.

avdv commented 1 week ago

I also see this inside of a nix-shell, where TMP and TMPDIR is set to a dedicated temp directory from /tmp/nix-shell.XXXXXX.

Can you provide more detailed instructions for reproducing this? I'd need the exact command line.

I have been trying to reproduce with:

TMPDIR=/home/oquenchil/tmpdir USE_BAZEL_VERSION=7.1.0 bazelisk build --action_env=TMPDIR=/home/oquenchil/tmpdir --sandbox_debug :foo

I think you need to set TMPDIR to a directory inside /tmp to trigger the error. Since /tmp is mounted empty inside the sandbox, the given TMPDIR does not exist...

It works for me, when I use --noincompatible_sandbox_hermetic_tmp.

\edit: Oh, I came here from #5900. I don't see the permission denied error, but No such file or directory.