search

NixOS / nixops

NixOps is a tool for deploying to NixOS machines in a network or cloud.
https://nixos.org/nixops
GNU Lesser General Public License v3.0
1.85k stars 366 forks source link

Why use ssh keyring when connecting to remotes? #593

Open 3noch opened 7 years ago

3noch commented 7 years ago

I had a scare recently when nixops suddenly stopped being able to connect to my servers. It turns out I had just recently added a new SSH key to my keyring (I'm one of those types who creates a new key pair for each host). The error was similar to #256. Of course, setting SSH_AUTH_SOCK='' resolves the issue. However, I'm wondering why on earth nixops wouldn't set this itself internally when connecting to servers? Since it's using SSH keys from the deployment state file, there is no reason to even touch the keyring.

edolstra commented 7 years ago

Could you provide more details on how to reproduce this issue? As far as I know, nixops doesn't interact with the SSH agent in any way...

globin commented 7 years ago

We actually exploit the fact that nixops uses the ssh agent so we can use our normal keys and don't have to share our state file.

When ssh-agent is running, adding keys with ssh-add should be enough to reproduce.

3noch commented 7 years ago

@globin That's very interesting. Can you explain how you do that? Do you bootstrap your servers with nixops as normal and then connect using an SSH key that can login as root? I didn't realize nixops was capable of doing deployments without the state file. I'm quite interested!

globin commented 7 years ago

We use the none backend and to install we build an iso, including our ssh keys and hydra and perform a normal installation with nixos-install with a key included for root and then deploy normally to that system.

3noch commented 7 years ago

@globin Fascinating! I'd love to read more about that approach if there is any.

3noch commented 7 years ago

Back on topic, it seems that not using the keyring at all would break at least some people's use case (e.g. @globin). I'm still curious to hear thoughts but that is at least 1 point against using something like SSH_AUTH_SOCK='' internally.

globin commented 7 years ago

See https://github.com/mayflower/hydra-jobs/blob/master/dist.nix for ISO/container tarball. Our nixops deployment isn't open source atm sadly, due to not having kept the secrets seperated cleanly, but we're planning to extract most of the stuff we can share again. Please ignore the channel in the hydra-jobs/dist.nix file, that stuff is broken currently.

3noch commented 7 years ago

I've since discovered that setting SSH_AUTH_SOCK='' will break any remote servers in NIX_REMOTE_SERVERS that require the keyring to access the key.

This is sounding more and more like a bad idea.