setupSecretsForUsers fails if ssh host keys are missing

[matrss]

I am running a NixOS system with tmpfs as root and use impermanence to provide ssh host keys at the usual location in /etc/ssh. The sops-nix secrets for users seem to be setup before that though, this is shown in journalctl:

stage-2-init: /nix/store/zx891jhqfmj32ca1sivi0cxl22007qkl-sops-install-secrets-0.0.1/bin/sops-install-secrets: Error setting up gpg keyring: Cannot read ssh key '/etc/ssh/ssh_host_rsa_key': open /etc/ssh/ssh_host_rsa_key: no such file or directory
stage-2-init: Activation script snippet 'setupSecretsForUsers' failed (1)

After booting, the directory in /var/run/secrets-for-users.d is empty.

The thing is that I am using a dedicated age key for sops-nix secrets and therefor did not expect sops-nix to fail.

To fix this I had to explicitly set sops.age.sshKeyPaths and sops.gnupg.sshKeyPaths to empty lists.

Intuitively I expected sops-nix to ignore missing ssh keys, especially if there is another key to use.

[Mic92]

For now you can disable it like this:

  sops.gnupg.sshKeyPaths = [];

[omernaveedxyz]

I am experiencing the same issue but in this case I do in fact use PGP keys for my machines and therefore I cannot use the workaround mentioned. Global secrets generate fine but the user secrets fail the same way that the original poster metioned.

Here are my logs for reference:

initrd-nixos-activation-start[392]: setting up secrets for users...
initrd-nixos-activation-start[413]: /nix/store/0f48rl4hhz08avfjxs5fbgy7gpy2wx66-sops-install-secrets-0.0.1/bin/sops-install-secrets: Error setting up gpg keyring: Cannot read ssh key '/etc/ssh/ssh_host_rsa_key': open /etc/ssh/ssh_host_rsa_key: no such file or directory
initrd-nixos-activation-start[392]: Activation script snippet 'setupSecretsForUsers' failed (1)
initrd-nixos-activation-start[428]: warning: password file ‘/run/secrets-for-users/users/omer/password’ does not exist
initrd-nixos-activation-start[392]: setting up /etc...
initrd-nixos-activation-start[392]: setting up secrets...
initrd-nixos-activation-start[567]: sops-install-secrets: Imported /etc/ssh/ssh_host_rsa_key as GPG key with fingerprint 5cd82e397a1c495419c0ef9635d756bf747bfc75
initrd-nixos-activation-start[567]: sops-install-secrets: Imported /etc/ssh/ssh_host_ed25519_key as age key with fingerprint age10sku4n4x9dqpvc4xsqy9u755vk90vnhvwm88x9l6upz3ewgslaesx9fwy5
systemd[1]: initrd-nixos-activation.service: Deactivated successfully.

I have tried to set the sops.gnupg.sshKeyPaths = [ '/persistent/etc/ssh/ssh_host_rsa_key' ] (pointing to the actual file location not the symlink) but this also does not work and results in the same error.

[omernaveedxyz]

I am experiencing the same issue but in this case I do in fact use PGP keys for my machines and therefore I cannot use the workaround mentioned. Global secrets generate fine but the user secrets fail the same way that the original poster metioned.

Here are my logs for reference:

initrd-nixos-activation-start[392]: setting up secrets for users...
initrd-nixos-activation-start[413]: /nix/store/0f48rl4hhz08avfjxs5fbgy7gpy2wx66-sops-install-secrets-0.0.1/bin/sops-install-secrets: Error setting up gpg keyring: Cannot read ssh key '/etc/ssh/ssh_host_rsa_key': open /etc/ssh/ssh_host_rsa_key: no such file or directory
initrd-nixos-activation-start[392]: Activation script snippet 'setupSecretsForUsers' failed (1)
initrd-nixos-activation-start[428]: warning: password file ‘/run/secrets-for-users/users/omer/password’ does not exist
initrd-nixos-activation-start[392]: setting up /etc...
initrd-nixos-activation-start[392]: setting up secrets...
initrd-nixos-activation-start[567]: sops-install-secrets: Imported /etc/ssh/ssh_host_rsa_key as GPG key with fingerprint 5cd82e397a1c495419c0ef9635d756bf747bfc75
initrd-nixos-activation-start[567]: sops-install-secrets: Imported /etc/ssh/ssh_host_ed25519_key as age key with fingerprint age10sku4n4x9dqpvc4xsqy9u755vk90vnhvwm88x9l6upz3ewgslaesx9fwy5
systemd[1]: initrd-nixos-activation.service: Deactivated successfully.

I have tried to set the sops.gnupg.sshKeyPaths = [ '/persistent/etc/ssh/ssh_host_rsa_key' ] (pointing to the actual file location not the symlink) but this also does not work and results in the same error.

This appears to have been fixed in the recent updates. Specifying sops.gnupg.sshKeyPaths correctly import the SSH keys from the persistent subvolume.

[WolfangAukang]

I had found a similar issue and according to journalctl -b, the bind mount for /etc goes after sops-nix, so if you were mounting your ssh keys through impermanence, sops-nix will fail because there is nothing in /etc yet. After referencing the keys on my /persist directory (what @omernaveedxyz proposed), it started working for me.

[tfkhim]

I faced the same issue as the initial poster while doing a fresh install with nixos-install.

The SSH host keys are generated as a preStart script in the sshd systemd unit. This script will only run at the first boot. Therefore sops-install-secrets fails during nixos-install.

I also use a dedicated age key. So setting sops.age.sshKeyPaths and sops.gnupg.sshKeyPaths to empty lists fixes the issue for me, too. Or I could just reboot twice in a row after the installation. The first reboot creates the host keys and the second reboot then works as expected.

But as long as there is no fix I would suggest to document this in the known limitations section of the Readme. It took me quite some time to figure this out. And others seem to have similar problems. What do you think?