DeterminateSystems / nix-installer

Install Nix and flakes with the fast and reliable Determinate Nix Installer, with over 7 million installs.
https://determinate.systems
GNU Lesser General Public License v2.1
2.28k stars 58 forks source link

EBPF installation issue when building multiarch Docker images #1305

Open heywoodlh opened 2 days ago

heywoodlh commented 2 days ago

I build my own Nix Docker image for various things. It's been building successfully for months now, but today, I ran into this issue:

22.67 Error: 
22.67    0: Install failure
22.67    1: Error executing action
22.67    2: Action `configure_nix` errored
22.67    3: Action `setup_default_profile` errored
22.67    4: Failed to execute command with status 1 `HOME="/root" NIX_SSL_CERT_FILE="/nix/store/9m6xbd8pcdb6c655b7lifhi5m1igi5rk-nss-cacert-3.101/etc/ssl/certs/ca-bundle.crt" "/nix/store/30gnc15nig1awa11vii9yz3z8518rnr3-nix-2.24.10/bin/nix-env" "--option" "substitute" "false" "-i" "/nix/store/30gnc15nig1awa11vii9yz3z8518rnr3-nix-2.24.10"`, stdout: 
22.67       stderr: installing 'nix-2.24.10'
22.67       error:
22.67              … while setting up the build environment
22.67 
22.67              error: unable to load seccomp BPF program: Invalid argument

I've had --extra-conf='filter-syscalls = false' per this comment: https://github.com/DeterminateSystems/nix-installer/issues/955#issuecomment-2099655058

It's been building successfully for months without issue -- however -- today when I attempt to build a multi-arch image -- it's almost like filter-syscalls = false is completely ignored on the foreign architecture.

Here's the steps to reproduce on an x86_64 Linux machine: ``` mkdir -p /tmp/testing && cd /tmp/testing curl -L https://gist.githubusercontent.com/heywoodlh/1f976c3912818dcd76f0cdf9e5fd3e32/raw/7f1b71c9d9eab3a363ad53e857a911ff92feff28/Dockerfile-det-sys-sample -o Dockerfile docker buildx create --name multiarch --driver docker-container --use docker buildx build --tag nix-testing --platform linux/arm64 . ``` However, the same Dockerfile builds normally on the same architecture: ``` ❯ docker build -t nix-testing . [+] Building 12.5s (6/6) FINISHED docker:rootless => [internal] load build definition from Dockerfile 0.0s => => transferring dockerfile: 343B 0.0s => [internal] load .dockerignore 0.0s => => transferring context: 2B 0.0s => [internal] load metadata for docker.io/library/alpine:latest 0.0s => CACHED [1/2] FROM docker.io/library/alpine:latest 0.0s => [2/2] RUN apk add --no-cache curl bash && curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | sh -s -- install linux --extra-conf "sandbox = false" --init none --no-con 11.8s => exporting to image 0.7s => => exporting layers 0.7s => => writing image sha256:3e9ad1f49aa7a33cbd29af5f0e513a36612dcd99b8183ea80c84d52f754e03fd 0.0s => => naming to docker.io/library/nix-testing ``` And here's a screenshot of my GitHub Actions running successfully until today :smile: ![image](https://github.com/user-attachments/assets/cce5a376-8ac5-44c1-8151-25bd0d797e28) And the failing log output on the ARM64 job -- the AMD64 job on the GitHub Action runner has no issue. ![image](https://github.com/user-attachments/assets/e92cfc81-4374-4b4a-88fe-7196d0ccea45)

Please let me know if I can provide any more details!

cole-h commented 10 hours ago

I think the issue is that we try to run all the Nix setup commands (like the nix-env command that is erroring) before we drop the Nix configuration... I can't think of a reason we would want to intentionally do that, so we can probably fix this by simply fixing the ordering to run the "place nix.conf" stuff before "run Nix setup commands" stuff.

cole-h commented 10 hours ago

I've opened https://github.com/DeterminateSystems/nix-installer/pull/1311 with that trivial fix and set it to upload -- once CI finishes, there are instructions in the PR body for trying it out, if you're so inclined!

EDIT: as of this edit, the binary is ready for testing:

curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix/pr/1311 | sh -s -- install