Open bbigras opened 2 years ago
Hit this today while trying to use Nix at work
Alright, I went down the rabbit hole on this and a number of things are involved in this failure.
Firstly, there's an issue where the default config of actions-runner-controller
does not lend itself to having user sandboxing in it's RunnerDeployment
. This is why they, by default, implement Docker support using a sidecar container. Maybe we could get the same thing to work by adding a sidecar that runs nix-daemon
and bind-mounting /nix
but that seemed like a lot of work.
The simple solution is to set the following in your RunnerDeployment
:
spec:
template:
spec:
image: "summerwind/actions-runner-dind"
dockerdWithinRunnerContainer: true
This will avoid the installer failing due to the sandbox not being able to work.
Secondly, there's the issue of USER
not being set. I'm not sure why it's like this, but you can work around this by adding it to the RunnerDeployment
env:
spec:
template:
spec:
env:
- name: USER
value: runner
Once the above are out of the way, we're still in trouble because the installation script here will trigger this bit of code: https://github.com/cachix/install-nix-action/blob/b2b9fc6cf9ad2446d49c06143ad34b98bf73be99/lib/install-nix.sh#L42-L48
And L46 there is, I believe, wrong on two levels:
-p
, so if you manually create the folder it explodesmkdir
in /etc
If you attempt to work around this by chmod
'ing /etc
you will break sudo
.
Finally, you can avoid using this action altogether and just manually doing what it would (kind of):
jobs:
test-nix:
runs-on: [self-hosted, linux, x64]
steps:
- name: install nix
run : |
sh <(curl -L https://nixos.org/nix/install) --no-daemon --no-channel-add
mkdir -p ~/.config/nix
touch ~/.config/nix/nix.conf
echo "max-jobs = auto" >> ~/.config/nix/nix.conf
echo "cores = 0" >> ~/.config/nix/nix.conf
echo "experimental-features = nix-command flakes" >> ~/.config/nix/nix.conf
echo "/nix/var/nix/profiles/per-user/$USER/profile/bin" >> "$GITHUB_PATH"
echo "/nix/var/nix/profiles/default/bin" >> "$GITHUB_PATH"
echo "NIX_PATH=nixpkgs=channel:nixos-unstable" >> "$GITHUB_ENV"
- name: nix-hello
run: nix run nixpkgs\#hello
@bbigras, can you try this out?
Oh, another thing, if you're running your cluster on GKE you need to force your node pool to use the ubuntu_containerd
image; I couldn't get cos_containerd
to work for some reason.
I'm still encountering this issue using cachix/install-nix-action@master
with #125 merged
<snip>/_actions/cachix/install-nix-action/master/install-nix.sh: line 20: USER: unbound variable
Any ideas?
Are you running those runners yourself?
Yeah I am, corporate restrictions means I need to self-host our runners. I do however have complete control over the runner image as it’s homebrewed.
I set the env: to have USER=runner like the other poster but it wasn’t happy still
Did you see the post above from @lovesegfault how to set $USER correctly?
I'm still having issues @domenkozar but these are relating to my custom runner image in use. I will leave this for now and revisit when I have some time to resolve this.
Did this ever get a confirmed resolution? We are also seeing this problem with runners deployed via the actions runner controller
@gthomson31 could you post the full log and other specific things about your environment?
We are also using the Actions-Runner-Controller to manage the runners - For the Runners we are building our own images built on top off
https://hub.docker.com/r/summerwind/actions-runner/tags
Have tested using images built from the following versions
Have not yet implemented the steps above mentioned to enabled dockerdWithinRunnerContainer
fix but going to give these a try and will feedback
FROM summerwind/actions-runner:latest
<REDACTED INTERNAL ENVS>
USER root
<REDACTED INTERNAL SCRIPTS>
Hashistack installation , etc
USER runner
Run cachix/install-nix-action@v19
Run /runner/_work/_actions/cachix/install-nix-action/v19/install-nix.sh
/runner/_work/_actions/cachix/install-nix-action/v19/install-nix.sh
shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
env:
INPUT_EXTRA_NIX_CONFIG:
INPUT_GITHUB_ACCESS_TOKEN:
INPUT_INSTALL_OPTIONS:
INPUT_INSTALL_URL:
INPUT_NIX_PATH:
GITHUB_TOKEN: ***
Installing Nix
/runner/_work/_actions/cachix/install-nix-action/v19/install-nix.sh: line 2[3](https://github.com/<redacted>/actions/runs/4162773921/jobs/7202357119#step:3:4): USER: unbound variable
Error: Process completed with exit code 1.
Also tried to set the ENV variable as part of the workflow run which threw up a new error.
name: Testing Nix
on: workflow_dispatch
env:
USER: "runner"
jobs:
example-job:
# Custom Runner Label - base-x.x.x
runs-on: base-1.0.0
steps:
- uses: cachix/install-nix-action@v19
Run cachix/install-nix-action@v19
with:
env:
USER: runner
Run /runner/_work/_actions/cachix/install-nix-action/v19/install-nix.sh
Installing Nix
installer options: --no-channel-add --darwin-use-unencrypted-nix-store-volume --nix-extra-conf-file /tmp/tmp.eTXGg50kNs/nix.conf
* Trying 34.141.28.239:443...
<REDACTED TLS HANDSHAKE>
* Connection #1 to host releases.nixos.org left intact
downloading Nix 2.13.2 binary tarball for x86_64-linux from 'https://releases.nixos.org/nix/nix-2.13.2/nix-2.13.2-x86_64-linux.tar.xz' to '/tmp/nix-binary-tarball-unpack.8CoAyqmlNS'...
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
100 20.0M 100 20.0M 0 0 29.8M 0 --:--:-- --:--:-- --:--:-- 29.8M
Note: a multi-user installation is possible. See https://nixos.org/manual/nix/stable/installation/installing-binary.html#multi-user-installation
Warning: the flag --darwin-use-unencrypted-nix-store-volume
is no longer needed and will be removed in the future.
performing a single-user installation of Nix...
directory /nix does not exist; creating it by running 'mkdir -m 0755 /nix && chown runner /nix' using sudo
copying Nix to /nix/store...
installing 'nix-2.13.2'
error: mounting /proc: Operation not permitted
(use '--show-trace' to show detailed location information)
/tmp/nix-binary-tarball-unpack.8CoAyqmlNS/unpack/nix-2.13.2-x86_64-linux/install: unable to install Nix into your default profile
Error: Process completed with exit code 1.
Could you try this workaround: https://github.com/NixOS/nix/issues/4107#issuecomment-710196475
@domenkozar trying to get this to work but it is not allowing me to unmount the directory within the summerwind image. Raising this with the project for support.
I'm trying to use install-nix action with https://github.com/actions-runner-controller/actions-runner-controller and I get:
If I set
USER
torunner
, I get permission error on/etc
. If I runsudo chmod 777 /etc
, it fails when trying to mount /proc.cc @hiroqn beacuse of https://github.com/cachix/install-nix-action/issues/43#issuecomment-890472909