Agenix symlinks incorrectly when using agenix secrets as initrd secrets

[Cu3PO42]

I'm currently attempting to setup a new server with NixOS. As part of my config, I have the following:

{
    // ...
    age.secrets."initrd_host_ed25519_key".file = ./secrets/initrd_host_ed25519_key.age;
    boot.initrd.network.ssh.hostKeys = [config.age.secrets."inintrd_host_ed25519_key".path];
}

This results in the following scenario: secrets now exist at /run/agenix/$generation/secret rather than /run/agenix/secret. I assume this is because the way initrd secrets are handled causes /run/agenix/initrd_host_ed25519_key to be created before agenix runs. Thus, when the symlink is created at age.nix Line 105, it is not created at ${cfg.secretsDir}, but rather inside that folder since it already exists.

I have worked around this problem by specifying a path outside /run/agenix for my hostkey only, but I believe it may be desirable to also include a if [ -L ${cfg.secretsDir} ]; then rm -rf ${cfg.secretsDir}; fi or similar instead of relying only on ln -f.

[pshirshov]

There might be a good solution for this problem:

  age.secrets.hostkey-initrd = {
    file = "${paths.agenix}/hostkey--initrd.age";
    path = "/etc/initrd-hostkey";
    symlink = false;
  };

...
boot.initrd.network.ssh.hostKeys = [ "/etc/initrd-hostkey" ];

Unfortunately this doesn't work because agenix sets up secrets during system activation stage but initrd is being built before that:

    system = {
      build.installBootLoader = finalSystemdBootBuilder;

Maybe agenix should use system.build instead of activation scripts.

Currently, as a dirty workaround, I deploy two times, first with boot.initrd.network.ssh.enable set to false (so I get the hostkey file copied) and then with true (so the initrd builder may pick up the hostkey).

@ryantm : do you think it may be possible to set up secrets BEFORE initrd is being built?..

[Cu3PO42]

I have my secret set up the same way you have @pshirshov. However, I never needed to deploy twice. I am currently using nixos-anywhere to provision servers and things have just worked (after modifying the secren setup like you have). From a cursory glance, nixos-anywhere doesn't appear to do any double activation (especially not with different configs!) so I don't know why this would work for me and not for you, unfortunately.

[pshirshov]

Maybe the initrd builder just reuses the file which persists between builds?

Obviously, if the hostkey persists between builds and reboots, you have to deploy twice only the first time and when the key changes.

That's acceptable but impure and fragile.

[pshirshov]

When it fails, the log looks like

updating systemd-boot from 253.3 to 253.6
cp: cannot stat '/etc/initrd-hostkey': No such file or directory
failed to create initrd secrets!
warning: error(s) occurred while switching to the new configuration

I'm using nixos-rebuild for deployments.

[Cu3PO42]

I have encountered that error as well. However that was only when I set up the key that agenix uses for decryption incorrectly. Ever since I fixed that error things have been working fine. nixos-anywhere builds the system closure and then simply calls into nixos-install with the provided closure.