Question about using btrfs-optin-persistence using systemd service

[omernaveedxyz]

I've been testing out using Systemd as the initrd system on NixOS (which at best seems to be in an alpha state...) and I've run into some issues/questions with the btrfs impermanence script that I'm hoping to resolve.

in btrfs-optin-persistence.nix, wiping the root partition is now done through a systemd service rather using the postDeviceCommands value for the custom NixOS initrd. That all makes sense to me. What I am wondering is how the service dependencies make sense.

requires = [ "initrd-root-device.target" ];
before = [ "sysroot.mount" ];
wantedBy = [ "initrd-root-fs.target" ];

When I tried this order, I encountered an error where initrd-btrfs-root-wipe would be run before the actual decryption was done for the root partion and therefore the service would fail.

I was able to resolve this, but the next issue was that sysroot.mount would activate while initrd-btrfs-root-wipe was still running and therefore fail again. To resolve these issues I ended up with something like this:

after = [ "systemd-cryptsetup@root.service" ];
before = [ "sysroot.mount" ];
wantedBy = [ "sysroot.mount" ];

PS: Thank you for having this as well as the nix-starter-config repository. They are by far the most helpful references I've encountered.

[Misterio77]

Sorry for the delay (life's been crazy)!

What I am wondering is how the service dependencies make sense.

I'm actually not using systemd in initrd for now, I've tried my best to make that unit work, but I always get the dependency cycles or race conditions :/

The main benefit it offers (for me, at least) is having plymouth start up before the LUKS password prompt, but it seems it's still missing a few features.

A fellow nixer (Tom) e-mailed me with a clever solution for this issue:

let
   rootfsDevice = "/dev/disk/by-label/${hostname}";
   wipeScript = ''
     counter=0

     # systemd initrd offers no way to hook in between
     # 'root device is ready' and 'mount the root device at /sysroot'.
     # Instead, lets poll for the root device.
     # https://github.com/systemd/systemd/issues/24904

     echo wipe: Waiting for ${rootfsDevice}
     while [ ! -e ${rootfsDevice} ]; do
       sleep 0.1
       counter=$((counter + 1))
       if [ $counter -ge 300 ]; then
           echo wipe: Timed out waiting for ${rootfsDevice}
           exit
       fi
     done
     echo wipe: Found ${rootfsDevice}. Wiping ephemeral.

     mkdir -p /btrfs
     mount -o subvol=/ ${rootfsDevice} /btrfs

     echo "Cleaning subvolume"
     btrfs subvolume list -o /btrfs/root | cut -f9 -d ' ' |
       while read subvolume; do
         btrfs subvolume delete "/btrfs/$subvolume"
       done && btrfs subvolume delete /btrfs/root

     echo "Restoring blank subvolume"
     btrfs subvolume snapshot /btrfs/root-blank /btrfs/root

     umount /btrfs
     rm -d /btrfs
   '';
in {
   options = {
     /* truncated */
   };
   config = {
     boot.initrd = {
       supportedFilesystems = [ "btrfs" ];
       systemd = {
         enable = true;
         emergencyAccess = true;
         initrdBin = with pkgs; [ coreutils btrfs-progs ];
         services.initrd-btrfs-root-wipe = {
           description = "Wipe ephemeral btrfs root";
           script = wipeScript;
           serviceConfig.Type = "oneshot";
           unitConfig.DefaultDependencies = "no";

           after = [ "cryptsetup.target" "initrd-root-device.target" ];
           before = [ "sysroot.mount" ];
           wantedBy = [ "sysroot.mount" ];
         };
       };
     };
   };

     /* truncated */
}

[Misterio77]

Thank you for having this as well as the nix-starter-config repository

Thanks a lot for your kind words! I'm very glad to be able to help out :)

[omernaveedxyz]

That solution seems similar to my own, but perhaps a little more janky. Anyway, thanks for getting back to me, good to know that my solution is reasonable.

[Misterio77]

I'm happy to say that I've finally found a elegant fix for this!

There's a .device unit for each and every device, so you can simply add the relevant one to your wipe service after/requires.

In my case, my btrfs partition is /dev/disk/by-lavel/${hostname}, so the relevant unit is dev-disk-by\x2dlabel-${hostname}.device (\x2d is how systemd escapes -). I found that out trying systemctl list-units -a | grep "dev-disk-by"

https://github.com/Misterio77/nix-config/blob/e0dda94d05fd506fd323fc0dc74ac93c591b057a/hosts/common/optional/ephemeral-btrfs.nix#L36

Note: if your unit has backslashes like mine, you have to either (if using regular nix strings) do double backslashes, or do the '' multiline string.

That means, "\foo" does not work, while "\\foo" and ''\foo'' does.

CC @omernaveedxyz