boulder: Create a shared compiler cache directory for rootless executions

[livingsilver94]

root is obviously risky. Building packages inherently allows for any code execution. While it's true that boulder runs the build in its own container, disabling root is another security improvement to prevent exploits I can't even predict at the moment. Plus, by just considering an unprivileged user, us developers won't have to think about where configuration files should be read from. Less complexity and more security. Win-win, in my opinion.

@ermo raised the concern that unprivileged users may not access a shared directory for compilation cache. Fortunately, ccache documents how to create a shared directory across users in a specific group (the builders group, for example). sccache can access a specific directory too, so we can reuse the same users+directory setup.

Demo

To demonstrate a shared directory works across unprivileged users, please find below a script that leverages toolbx to create a shared build environment. Note that toolbx is not required for such setup, I'm just using it to create a demo environment without cluttering my/your system. Please copy and paste the script line by line; a global copy-paste won't work because we're entering and exiting containers and subshells.

# Create the testing container and enter it.
toolbox create --distro fedora sccache -y
toolbox enter sccache

# Prepare sccache and the compiler toolchain.
sudo dnf install -y cargo cmake make gcc g++ openssl-devel
cargo install sccache --locked
sudo cp "/home/$USER/.cargo/bin/sccache" /

# Create builders.
sudo groupadd sccache
sudo useradd --groups sccache builder1
sudo useradd --groups sccache builder2

# Set up a shared directory for cache per https://ccache.dev/manual/latest.html#_sharing_a_local_cache
# Similarly, we want to create a shared dir from which to fetch sources. sccache only caches files if their
# absolute paths match. This shouldn't be an issue in production, since boulder would just use its own directory.
sudo mkdir /var/cache/sccache
sudo chown root:sccache /var/cache/sccache
sudo chmod 2775 /var/cache/sccache
sudo chmod g+s /var/cache/sccache
sudo mkdir /var/lib/build
sudo chown root:sccache /var/lib/build
sudo chmod 2775 /var/lib/build
sudo chmod g+s /var/lib/build

# Try to build as two different users.
# The umask will have to be set permanently for each user in the sccache group.
# Otherwise, each user will prevent other users from *writing* files in the shared directories.
sudo --login --user=builder1
umask 0002
export SCCACHE_DIR=/var/cache/sccache
export SCCACHE_NO_DAEMON=1
git clone https://github.com/memsharded/hello /var/lib/build/hello
cmake -S /var/lib/build/hello/ -B /var/lib/build/hello/$USER -DCMAKE_CXX_COMPILER_LAUNCHER=/sccache
make -C /var/lib/build/hello/$USER
exit
sudo --login --user=builder2
umask 0002
export SCCACHE_DIR=/var/cache/sccache
export SCCACHE_NO_DAEMON=1
cmake -S /var/lib/build/hello/ -B /var/lib/build/hello/$USER -DCMAKE_CXX_COMPILER_LAUNCHER=/sccache
make -C /var/lib/build/hello/$USER
exit

[ermo]

One way to make this nice on Serpent is to have a trigger that auto-adds users who are in wheel and who have a GID >= 1000 to the cache dir permissions (I think we're looking at extended ACLs for this).

The idea could be to run a boulder command that does this for the relevant users; then we use the same code path whether we're on serpent or something else; it's just that on serpent, we have a moss trigger manage the call.

[livingsilver94]

I have some questions:

• Are we automatically adding users belonging to sudo and wheel to the new builder group?
• We should make boulder remember that it should use the shared ccache directory path, so we need to edit a config file. What should be the path of such file? NOTE: I think boulder uses the config file in $HOME by default.
• Are we going to allow to add users to the builder group later on (that is, when we already set it up)?
• Should we allow to customize the group name and the cache directory path, via CLI?