search

rundeck / rundeck

Enable Self-Service Operations: Give specific users access to your existing tools, services, and scripts
http://rundeck.org
Apache License 2.0
5.46k stars 908 forks source link

SSH executor is failing with Valid SSH Key - "key type ssh-rsa not in PubkeyAcceptedAlgorithms" #7815

Open dev-rowbot opened 2 years ago

dev-rowbot commented 2 years ago

Describe the bug I am running Rundeck 4.2.1 in Kubernetes and when I try execute a job using the SSH connector I get the error

Failed: AuthenticationFailure: Authentication failure connecting to node: "bastion". Could not authenticate.

I know the SSH key works because I can ssh from Kubernetes pod to the bastion server using the same key. I did have an issue with the key previously due to the format but I regenerated the key using this command

ssh-keygen -p -m PEM -f .ssh/id_rsa

The full log is as follows:

Using ssh keyfile: /home/rundeck/.ssh/id_rsa
Starting SSH Connection: ubuntu@10.50.20.230 (bastion)
Using private key file: /home/rundeck/.ssh/id_rsa
Set timeout to 0
Connecting to 10.50.20.230:22
Connecting to 10.50.20.230 port 22
Connection established
Remote version string: SSH-2.0-OpenSSH_8.9p1 Ubuntu-3
Local version string: SSH-2.0-JSCH-0.1.54
CheckCiphers: aes256-ctr,aes192-ctr,aes128-ctr,aes256-cbc,aes192-cbc,aes128-cbc,3des-ctr,arcfour,arcfour128,arcfour256
CheckKexes: diffie-hellman-group14-sha1,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521
CheckSignatures: ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521
SSH_MSG_KEXINIT sent
SSH_MSG_KEXINIT received
kex: server: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,sntrup761x25519-sha512@openssh.com,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256
kex: server: rsa-sha2-512,rsa-sha2-256,ecdsa-sha2-nistp256,ssh-ed25519
kex: server: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
kex: server: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com
kex: server: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
kex: server: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
kex: server: none,zlib@openssh.com
kex: server: none,zlib@openssh.com
kex: server: 
kex: server: 
kex: client: ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group14-sha1,diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group1-sha1
kex: client: ssh-rsa,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521
kex: client: aes128-ctr,aes128-cbc,3des-ctr,3des-cbc,blowfish-cbc,aes192-ctr,aes192-cbc,aes256-ctr,aes256-cbc
kex: client: aes128-ctr,aes128-cbc,3des-ctr,3des-cbc,blowfish-cbc,aes192-ctr,aes192-cbc,aes256-ctr,aes256-cbc
kex: client: hmac-md5,hmac-sha1,hmac-sha2-256,hmac-sha1-96,hmac-md5-96
kex: client: hmac-md5,hmac-sha1,hmac-sha2-256,hmac-sha1-96,hmac-md5-96
kex: client: none
kex: client: none
kex: client: 
kex: client: 
kex: server->client aes128-ctr hmac-sha1 none
kex: client->server aes128-ctr hmac-sha1 none
SSH_MSG_KEX_ECDH_INIT sent
expecting SSH_MSG_KEX_ECDH_REPLY
Permanently added '10.50.20.230' (ECDSA) to the list of known hosts.
SSH_MSG_NEWKEYS sent
SSH_MSG_NEWKEYS received
SSH_MSG_SERVICE_REQUEST sent
SSH_MSG_SERVICE_ACCEPT received
Authentications that can continue: publickey
Next authentication method: publickey
Disconnecting from 10.50.20.230 port 22
SSH command execution error: AuthenticationFailure: Authentication failure connecting to node: "bastion". Could not authenticate.
Failed: AuthenticationFailure: Authentication failure connecting to node: "bastion". Could not authenticate.

I added this to my project config to try force publickey and disable strict host key checking

project.ssh-config-PreferredAuthentications=publickey
project.ssh-config-StrictHostKeyChecking=no

The error is caused by the version of sshd running on my Ubuntu instance (22.04). In the /var/log/auth.log I noticed this

Jul 26 13:50:42 ip-10-50-20-230 sshd[58329]: userauth_pubkey: key type ssh-rsa not in PubkeyAcceptedAlgorithms [preauth]

The fix for the issue is to add this to /etc/ssh/sshd_config

PubkeyAuthentication yes
PubkeyAcceptedKeyTypes=+ssh-rsa

My Rundeck detail

To Reproduce As per the bug description

  1. setup a new Ubuntu 22.04 Server Instance.
  2. Create RSA Key for Rundeck and add the publci key to the Ubuntu servers authorized_keys file
  3. Add the node config to Rundeck and specify the ssh key to use in the project
  4. Run a job and attempt to use node

Expected behavior SSh Should work without any changes required in the sshd_config file in ubuntu.

mirkoknoll commented 1 year ago

Got same issue with upgraded node to Ubuntu 22.04. Node was working without issues on Ubuntu 20.04, last successful job right before the upgrade. Checking the same job after upgrade was failing. Access to node via ssh works from Rundeck server (Rundeck 4.5.0), but not via job. Adding suggested fix in sshd_config solved the issue for me. Thanks @dev-rowbot

jsboak commented 1 year ago

@dev-rowbot have you tried using the SSHJ Node executor plugin?

https://github.com/rundeck-plugins/sshj-plugin#sshj-plugin-rundeck-plugin

leonboot commented 11 months ago

Since we've been upgrading our servers to Debian 12 we have been seeing this error more and more. It seems Debian 12 has dropped support for RSA keys by default, which I believe is a sensible thing to do as using RSA keys is increasingly discouraged. As Rundeck only supports RSA and DSA key types, this requires negating this security measure by re-enabling support for RSA keys.

Supporting ED25519 would certainly be a welcome update to Rundeck!

Matthew-Jenkins commented 11 months ago

I've been thinking of asking for rundeck enterprise where I work. I use a community version of rundeck for just a few restart jobs that unskilled people need to occasionally 'click the button'. But if there is no support for anything besides rsa there is no point.

rsa is deprecated in rhel8 and disabled for small key completely. rsa is completely disabled in rhel9.

Get off your butts and get ed25519 support before other companies evaluating rundeck decide to not use it.

MegaDrive68k commented 11 months ago

Hi @Matthew-Jenkins ,

The SSHJ node executor supports ed25519 keys. Please change the project node executor to SSHJ.

ledkins commented 6 months ago

Hi, I have also hit this same problem where I have a rundeck system setup on linux 7.9 and we have a couple of new linux 9 VM's which I need to introduce into rundeck. Hitting the same error: userauth_pubkey: key type ssh-rsa not in PubkeyAcceptedAlgorithms [preauth] Before I go to the trouble of downloading and troubleshooting the installation of the SSHJ plugin, can you please confirm this is the correct solution? There is not much doc about how to setup the plugin. Any advice would be appreciated. I plan on setting up a separate project for all jobs to run against the linux 9 VM's so I can configure the project to use this plugin. I have no idea if I am on the right path here. Regards Lorraine

leonboot commented 6 months ago

The SSHJ plugin, which is the default SSH executor now, does support modern key types like ed25519. These types are preferred by the more recent versions of Linux distros, so yes, this would fix the issue. However, there is a bug in the new SSHJ-SCP file copier that has yet to be fixed: https://github.com/rundeck/rundeck/issues/8507. For me, this is a show stopper as it breaks a lot of our scripts. It might not affect your installation, however.