Open oluceps opened 5 months ago
To be mentioned that when system.etc.overlay.mutable = false;
this does not appear.
@nikstur this happens for me also when I have anything mounted on top of /etc.
This reproduces it:
diff --git a/nixos/tests/activation/etc-overlay-mutable.nix b/nixos/tests/activation/etc-overlay-mutable.nix
index 087c06408a71..5b150c61b08f 100644
--- a/nixos/tests/activation/etc-overlay-mutable.nix
+++ b/nixos/tests/activation/etc-overlay-mutable.nix
@@ -25,6 +25,9 @@
machine.succeed("/run/current-system/bin/switch-to-configuration test")
with subtest("switching to a new generation"):
+ machine.succeed("mkdir /etc/mountpoint")
+ machine.succeed("mount -t tmpfs tmpfs /etc/mountpoint")
+
machine.fail("stat /etc/newgen")
machine.succeed("echo -n 'mutable' > /etc/mutable")
Still experiencing
kaambl | Evaluated kaambl
kaambl | Building kaambl
kaambl | /nix/store/ccjy49f1x5gvdgsb9qmi7crl2n05hisj-nixos-system-kaambl-24.11.20240729.9f10e67
kaambl | Built "/nix/store/ccjy49f1x5gvdgsb9qmi7crl2n05hisj-nixos-system-kaambl-24.11.20240729.9f10e67"
kaambl | Pushing system closure
kaambl | Pushed system closure
kaambl | No pre-activation keys to upload
kaambl | Activating system profile
kaambl | Installing Lanzaboote to "/efi"...
kaambl | Collecting garbage...
kaambl | Successfully installed Lanzaboote.
kaambl | stopping the following units: systemd-modules-load.service, systemd-tmpfiles-resetup.service, systemd-udevd-control.socket, systemd-udevd-kernel.socket, systemd-udevd.service
kaambl | activating the configuration...
kaambl | remounting /etc...
kaambl | mount: /tmp/tmp.uvOanRgx7X: /dev/loop0 already mounted or mount point busy.
kaambl | dmesg(1) may have more information after failed mount system call.
kaambl | Moving mount
kaambl | Mounting beneath top mount
kaambl | Attaching mount /tmp/tmp.ShpAHKUkAh -> /etc
kaambl | Moving single attached mount
kaambl | Activation script snippet 'etc' failed (32)
kaambl | Failed to run activate script
kaambl | reloading user units for elen...
kaambl | Error: Failed to restart nixos-activation.service
kaambl |
kaambl | Caused by:
kaambl | Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.
kaambl | restarting sysinit-reactivation.target
kaambl | Failed to restart sysinit-reactivation.target: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.
kaambl | Error: Failed to get unit dbus-broker.service
kaambl |
kaambl | Caused by:
kaambl | Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.
> nix-info -m
- system: `"x86_64-linux"`
- host os: `Linux 6.10.2, NixOS, 24.11 (Vicuna), 24.11.20240729.9f10e67`
- multi-user?: `yes`
- sandbox: `yes`
- version: `nix-env (Lix, like Nix) 2.91.0-dev-pre20240726-6abad7c
System type: x86_64-linux
Additional system types: i686-linux, x86_64-v1-linux, x86_64-v2-linux, x86_64-v3-linux
Features: gc, signed-caches
System configuration file: /etc/nix/nix.conf
User configuration files: /home/elen/.config/nix/nix.conf:/etc/xdg/nix/nix.conf:/home/elen/.local/share/flatpak/exports/etc/xdg/nix/nix.conf:/var/lib/flatpak/exports/etc/xdg/nix/nix.conf:/home/elen/.nix-profile/etc/xdg/nix/nix.conf:/nix/profile/etc/xdg/nix/nix.conf:/home/elen/.local/state/nix/profile/etc/xdg/nix/nix.conf:/etc/profiles/per-user/elen/etc/xdg/nix/nix.conf:/nix/var/nix/profiles/default/etc/xdg/nix/nix.conf:/run/current-system/sw/etc/xdg/nix/nix.conf
Store directory: /nix/store
State directory: /nix/var/nix
Data directory: /nix/store/w1y9gd6yxf8azq4ilnk7ghcbjkcp2bbx-lix-2.91.0-dev-pre20240726-6abad7c/share`
- nixpkgs: `/nix/store/n5yzhgbv2vrf43rjdw831xynv82by12f-rb49nm580v5dp49y1ram2byyg7pd4sj1-source`
Can you provide kernel logs with dmesg
for this mount call? Otherwise I cannot tell what's going on. It looks like it fails to mount the metadata image.
Can you provide kernel logs with
dmesg
for this mount call? Otherwise I cannot tell what's going on. It looks like it fails to mount the metadata image.
I haven't seen any kernel log related to this, and it's hard to reproduce. I'll stay in this nixpkgs revision for weeks to see if I can reproduce it.
journalctl -k -p 5 --since 14:00
https://pb.nyaw.xyz/on-toucan.txt
Maybe? related https://github.com/NixOS/nixpkgs/issues/333999
FWIW I set system.switch.enableNg = true;
https://github.com/oluceps/nixos-config/blob/e0f6880a135dcc20c02f5452d384376a0141bf80/misc.nix#L13-L14
@nikstur I don't think there will be any logs if the kernel returns -EBUSY on the mount syscall. I think mount only prints this for filesystems that have custom error logs.
@oluceps can you run strace -f -s512 -e mount nixos-rebuild switch
as root? And give us the output?
@oluceps can you run
strace -f -s512 -e mount nixos-rebuild switch
as root? And give us the output?
Here's the log, sudo strace -f -s512 -e mount nixos-rebuild switch --flake .
https://pb.nyaw.xyz/famous-squirrel.txt
Interesting. This looks different than expected. I would have expected a mount system call, but it seemed to have failed in a different syscall.
Describe the bug
Cannot remount /etc while rebuilding with
etc.overlay
enabled. This issue does not always appear.May related: https://github.com/NixOS/nixpkgs/pull/270727 https://github.com/NixOS/nixpkgs/issues/291398
Steps To Reproduce
kernel log:
Expected behavior
/etc
successfully mounted and switch complete.Screenshots
If applicable, add screenshots to help explain your problem.
Additional context
Add any other context about the problem here.
Notify maintainers
Metadata
Please run
nix-shell -p nix-info --run "nix-info -m"
and paste the result.Add a :+1: reaction to issues you find important.