Failed to get a signed key from the CA: timed out while waiting for a response from the CA

[schlos]

I'm failing to connect to bot-sshca enabled server. See below:

Local terminal:

$ kssh developer@xxx.xxx.xxx.xxx
Failed to get a signed key from the CA: timed out while waiting for a response from the CA

Keybase SSH Provision chat in my team:

ME: AckRequest--my_keybase_username

BOT: Ack--my_keybase_username

ME: AckRequest--my_keybase_username

BOT: Ack--my_keybase_username

ME: Signature_Request:{"ssh_public_key":"ssh-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/h+xxxxxxxxx local_user@computer-name.local\n","uuid":"xxxxxxxxx-xxxxx-xxxx-xxxxxx-xxxxxxxxxxx"}

BOT: Signature_Response:{"signed_key":"ssh-ed25519-cert-v01@openssh.com SOME_LONG_KEY local_user@computer-name.local\n","uuid":"xxxxxxxxx-xxxxx-xxxx-xxxxxx-xxxxxxxxxxx"}

On Keybase chat it looks as SSH key was properly provided. So I tried again.

Local terminal:

$ kssh developer@xxx.xxx.xxx.xxx
developer@xxx.xxx.xxx.xxx: Permission denied (publickey).
SSH exited with err: exit status 255

Does anybody have idea what could be a problem?

[ddworken]

It looks like the first issue was a timeout where the reply from the bot was right after the timeout period (which I have never seen before, so you seem to just have gotten really unlucky! But please do let me know if it happens again).

For the second issue, that likely means that the ssh server you were trying to access refused the connection. Did you follow all the steps for the server set up?

In terms of debugging, can you share with me the output of running the below commands on the server you are trying to access:

cat /etc/ssh/ca.pub
cat /etc/ssh/auth_principals/developer
cat /etc/ssh/sshd_config

/etc/ssh/ca.pub should have the fingerprint of the public key (given to you when you ran make generate). /etc/ssh/auth_principals/developer should have the Keybase teams that you wish to grant access in it (eg teamname.ssh.prod). /etc/ssh/sshd_config should have the below two lines somewhere in it:

TrustedUserCAKeys /etc/ssh/ca.pub
AuthorizedPrincipalsFile /etc/ssh/auth_principals/%u

Also, feel free to ping me on Keybase (keybase.io/dworken) if you want to chat about this or have any other questions.

[bijeebuss]

I had basically the same thing happen cat /etc/ssh/ca.pub

ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIKjX/oXgi8eHLC+4lYnweLH42NIMTgTS2JiPGKWIxPqP keybase@ee47cdb2a0c8

cat /etc/ssh/auth_principals/developer

bijeebusca.ssh.root

cat /etc/ssh/sshd_config

# Package generated configuration file
# See the sshd_config(5) manpage for details

# What ports, IPs and protocols we listen for
Port 22
# Use these options to restrict which interfaces/protocols sshd will bind to
#ListenAddress ::
#ListenAddress 0.0.0.0
Protocol 2
# HostKeys for protocol version 2
HostKey /etc/ssh/ssh_host_rsa_key
HostKey /etc/ssh/ssh_host_dsa_key
HostKey /etc/ssh/ssh_host_ecdsa_key
HostKey /etc/ssh/ssh_host_ed25519_key
#Privilege Separation is turned on for security
UsePrivilegeSeparation yes

# Lifetime and size of ephemeral version 1 server key
KeyRegenerationInterval 3600
ServerKeyBits 1024

# Logging
SyslogFacility AUTH
LogLevel INFO

# Authentication:
LoginGraceTime 120
PermitRootLogin prohibit-password
StrictModes yes

RSAAuthentication yes
PubkeyAuthentication yes
AuthorizedKeysFile   /srv/authorized_keys/%u/authorized_keys

# Don't read the user's ~/.rhosts and ~/.shosts files
IgnoreRhosts yes
# For this to work you will also need host keys in /etc/ssh_known_hosts
RhostsRSAAuthentication no
# similar for protocol version 2
HostbasedAuthentication no
# Uncomment if you don't trust ~/.ssh/known_hosts for RhostsRSAAuthentication
#IgnoreUserKnownHosts yes

# To enable empty passwords, change to yes (NOT RECOMMENDED)
PermitEmptyPasswords no

# Change to yes to enable challenge-response passwords (beware issues with
# some PAM modules and threads)
ChallengeResponseAuthentication no

# Change to no to disable tunnelled clear text passwords
PasswordAuthentication no

# Kerberos options
#KerberosAuthentication no
#KerberosGetAFSToken no
#KerberosOrLocalPasswd yes
#KerberosTicketCleanup yes

# GSSAPI options
#GSSAPIAuthentication no
#GSSAPICleanupCredentials yes

X11Forwarding yes
X11DisplayOffset 10
PrintMotd no
PrintLastLog yes
TCPKeepAlive yes
#UseLogin no

#MaxStartups 10:30:60
#Banner /etc/issue.net

# Allow client to pass locale environment variables
AcceptEnv LANG LC_*

Subsystem sftp /usr/lib/openssh/sftp-server

# Set this to 'yes' to enable PAM authentication, account processing,
# and session processing. If this is enabled, PAM authentication will
# be allowed through the ChallengeResponseAuthentication and
# PasswordAuthentication.  Depending on your PAM configuration,
# PAM authentication via ChallengeResponseAuthentication may bypass
# the setting of "PermitRootLogin without-password".
# If you just want the PAM account and session checks to run without
# PAM authentication, then enable this but set PasswordAuthentication
# and ChallengeResponseAuthentication to 'no'.
# UsePAM yes

TrustedUserCAKeys /etc/ssh/ca.pub
AuthorizedPrincipalsFile /etc/ssh/auth_principals/%u

[ddworken]

@bijeebuss Hmm, and to confirm command you are running is:

kssh developer@x.x.x.x

Can you also share the output of:

ssh-keygen -L -f ~/.ssh/keybase-signed-key---cert.pub

This will confirm that the certificate you provisioned has the correct teams stored in it and that it is signed with the correct key.

[bijeebuss]

        Type: ssh-ed25519-cert-v01@openssh.com user certificate
        Public key: ED25519-CERT SHA256:hdZTSGEUIRm9808qEfBv5/XTvGH2vUokq6FUrUfrWSo
        Signing CA: ED25519 SHA256:62UsnfZYBSOrpmNvDfgLcOT1gpHCiwKyhpdgwm86iV4
        Key ID: "b874e27d-41a5-4c85-ba03-e8328a25d716:3d962cf4-620e-47e5-ad84-cc33be409000:bijeebus"
        Serial: 0
        Valid: from 2019-09-12T09:13:00 to 2019-09-12T10:14:53
        Principals: 
                bijeebusca.ssh.root
        Critical Options: (none)
        Extensions: 
                permit-X11-forwarding
                permit-agent-forwarding
                permit-port-forwarding
                permit-pty
                permit-user-rc

[bijeebuss]

yes that's the command I'm using, tried root as well

[RayOei]

I regularly have this issue as well: Failed to get a signed key from the CA: timed out while waiting for a response from the CA

In the Keybase chat I can see the certificate is provided: User request:

AckRequest--user
AckRequest--user
< a lot of those>
AckRequest--user
AckRequest--user
AckRequest--user
Signature_Request:{"ssh_public_key":"ssh-ed25519 ...blabla

And the bot is answering:

Ack--user
Ack--user
Ack--user
Ack--user
< a lot more of those>
Ack--user
Ack--user
Signature_Response:{"signed_key":"ssh-ed25519-cert-v01@openssh.com AAAAIHNzaC1lZDI1NTE5LWNlcnQtdjAxQG9wZW5zc2guY29tAAAAINHMjtvb+uIOhPvBa73oFyDihX6OEHqdeM70Wr+bGScIAAAAIGU0vd2DbkeQJh8+nzMryEHV/OrKiyDKpi0hDT+z6sblablablabla

In the mean time kssh has failed with the error.

I have to retry, sometimes up to 4 times before I get a connection. So it seems either the timeout on the kssh side is too strict or the bot reacts a bit too slow? When the connection is successful I only see the user request and the signature response from the bot and single Ackrequest and Ack messages....