Mounting paths needed for boot can fail

[m-bdf]

Booting a system with environment.persistence."/persistent".directories = [ "/var/lib/nixos" ] (from the README) fails if /persistent/var/lib/nixos does not exist yet:

waiting for device /mnt-root/persistent/var/lib/nixos to appear.......................
Timed out waiting for device /mnt-root/persistent/var/lib/nixos, trying to mount anyway.
mounting /mnt-root/persistent/var/lib/nixos on /var/lib/nixos...
mount: mounting /mnt-root/persistent/var/lib/nixos on /mnt-root/var/lib/nixos failed: No such file or directory

An error occurred in stage 1 of the boot process, which must mount the
root filesystem on `/mnt-root` and then start stage 2. Press one
of the following keys:

  r) to reboot immediately
  *) to ignore the error and continue

The init script of boot stage 1 tries to mount non-existent /persistent/var/lib/nixos on /var/lib/nixos, because this path is needed for boot. However, the activation script responsible for creating missing directories is only run after stage 1, so mounting fails.

This issue should occur for any path needed for boot, whose persistent storage does not exist at boot.

[m-bdf]

Note that for some reason this is not an issue with boot.initrd.systemd.enable = true;

[Akimitsu333]

Maybe you need the option neededForBoot.

[m-bdf]

I agree it looks like this should fix it, but unfortunately it doesn't, my /persistent filesystem is already marked neededForBoot. If it works with systemd-based initrd, maybe it's a bug in NixOS's traditional scripted initrd?

[MrFoxPro]

For anyone who struggles with this, try:

boot.initrd.systemd.enable = true;
boot.loader.efi.canTouchEfiVariables = true;

[talyz]

Please try #172 and see if it fixes this issue for you.

[visualphoenix]

Also hitting this with nixos-anywhere and disko - commenting out impermanence allows nixos-anywhere to correctly provision.

with impermanence enabled, doing a clean nixos-anywhere flake based install to a remote host using disko fails:

copying path '/nix/store/i8a31irgk06kr01jndjk9dzjwvfknb2r-unit-suid-sgid-wrappers.service' from 'https://cache.nixos.org'...
copying path '/nix/store/bqn3fchv6v8c7ym7zkvn0ks6k66mkc33-linux-6.6.34-modules' from 'https://cache.nixos.org'...
### Installing NixOS ###
Pseudo-terminal will not be allocated because stdin is not a terminal.
Warning: Permanently added 'xxx.xxx.xxx.xxx' (ED25519) to the list of known hosts.
installing the boot loader...
Warning: Source directory '/nix/persist/system/etc' does not exist; it will be created for you with the following permissions: owner: 'root:root', mode: '0755'.
mkdir: cannot create directory ‘/nix/persist/system/etc’: No such file or directory
...
FileNotFoundError: [Errno 2] No such file or directory: '/etc/machine-id'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/nix/store/1lksf0kkffcnw5l8ryq5imai8pdlpy13-bza6dmx1w5c0xrvs1m7704ijnzqcrsfi-systemd-boot", line 394, in <module>
    main()
  File "/nix/store/1lksf0kkffcnw5l8ryq5imai8pdlpy13-bza6dmx1w5c0xrvs1m7704ijnzqcrsfi-systemd-boot", line 377, in main
    install_bootloader(args)
  File "/nix/store/1lksf0kkffcnw5l8ryq5imai8pdlpy13-bza6dmx1w5c0xrvs1m7704ijnzqcrsfi-systemd-boot", line 267, in install_bootloader
    machine_id = subprocess.run(
                 ^^^^^^^^^^^^^^^
  File "/nix/store/7hnr99nxrd2aw6lghybqdmkckq60j6l9-python3-3.11.9/lib/python3.11/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/nix/store/gv0jmdv734pdxg6ilb4kq2np2fxxkr39-systemd-255.6/bin/systemd-machine-id-setup', '--print']' returned non-zero exit status 1.
installation finished!
umount: /mnt/boot unmounted
umount: /mnt/nix unmounted
umount: /mnt unmounted
### Waiting for the machine to become reachable again ###
ssh: connect to host xxx.xxx.xxx.xxx port 22: Connection refused
### Done! ###

i tried #172 but it didnt help.

i have the flags enabled:

boot.initrd.systemd.enable = true;
boot.loader.efi.canTouchEfiVariables = true;

i confirmed the partitions are marked neededForBoot = true;

[Curious-r]

@visualphoenix Did you mean it can not find any boot device after reboot? Just to provide a little inspiration: After installed with nixos-anywhere and impermanece, did you checked if the ESP partation be written actually? If not, maybe you can try to leave 1M before ESP partation in disko config.