Unable to identify the correct private key for the corresponding security key

[zanda8893]

When passing multiple private keys for security keys to asyncssh.connect using client_keys an exception is throw if the first key checked is not correct for the security key.

Traceback (most recent call last):
  File "/home/zanda/new_tools/fstn_tools/automated_vm_connection/fstn_remote/components/ssh.py", line 367, in async_connect_forward
    await connect('proxmoxJump', username, password)
  File "/home/zanda/new_tools/fstn_tools/automated_vm_connection/fstn_remote/components/ssh.py", line 312, in connect
    return await ssh_obj.connect(host)
  File "/home/zanda/new_tools/fstn_tools/automated_vm_connection/fstn_remote/components/ssh.py", line 189, in connect
    await self.connect(host.proxy_host)  # Wait for connection
  File "/home/zanda/new_tools/fstn_tools/automated_vm_connection/fstn_remote/components/ssh.py", line 192, in connect
    return await self._establish_connection(host, tunnel)
  File "/home/zanda/new_tools/fstn_tools/automated_vm_connection/fstn_remote/components/ssh.py", line 227, in _establish_connection
    conn = await asyncssh.connect(
  File "/home/zanda/new_tools/fstn_tools/automated_vm_connection/.venv/lib/python3.12/site-packages/asyncssh/connection.py", line 8834, in connect
    return await asyncio.wait_for(
  File "/usr/lib/python3.12/asyncio/tasks.py", line 520, in wait_for
    return await fut
  File "/home/zanda/new_tools/fstn_tools/automated_vm_connection/.venv/lib/python3.12/site-packages/asyncssh/connection.py", line 453, in _connect
    await options.waiter
  File "/home/zanda/new_tools/fstn_tools/automated_vm_connection/.venv/lib/python3.12/site-packages/asyncssh/connection.py", line 1092, in _reap_task
    task.result()
  File "/home/zanda/new_tools/fstn_tools/automated_vm_connection/.venv/lib/python3.12/site-packages/asyncssh/auth.py", line 343, in _send_signed_request
    await self.send_request(Boolean(True),
  File "/home/zanda/new_tools/fstn_tools/automated_vm_connection/.venv/lib/python3.12/site-packages/asyncssh/auth.py", line 136, in send_request
    await self._conn.send_userauth_request(self._method, *args, key=key)
  File "/home/zanda/new_tools/fstn_tools/automated_vm_connection/.venv/lib/python3.12/site-packages/asyncssh/connection.py", line 1948, in send_userauth_request
    sig = await self._loop.run_in_executor(None, key.sign, data)
  File "/usr/lib/python3.12/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/home/zanda/new_tools/fstn_tools/automated_vm_connection/.venv/lib/python3.12/site-packages/asyncssh/public_key.py", line 2271, in sign
    return self._key.sign(data, self.sig_algorithm)
  File "/home/zanda/new_tools/fstn_tools/automated_vm_connection/.venv/lib/python3.12/site-packages/asyncssh/public_key.py", line 569, in sign
    self.sign_ssh(data, sig_algorithm)))
  File "/home/zanda/new_tools/fstn_tools/automated_vm_connection/.venv/lib/python3.12/site-packages/asyncssh/sk_ecdsa.py", line 172, in sign_ssh
    flags, counter, sig = sk_sign(sha256(data).digest(), self._application)
  File "/home/zanda/new_tools/fstn_tools/automated_vm_connection/.venv/lib/python3.12/site-packages/asyncssh/sk.py", line 207, in sk_sign
    raise ValueError('Security key credential not found')

ValueError: Security key credential not found

Is there anyway to pass multiple sk private key files and have the correct one selected instead of throwing an exception?

[ronf]

The "Security key credential not found" error generally means that the security key you are trying to use is not currently attached to the system. If you have multiple security keys plugged in simultaneously, it should automatically try all of the available keys, generating hat error only when it runs out of keys to try (or if no keys were attached).

Keep in mind that AsyncSSH doesn't prompt for you to plug in a key. It expects it to be already plugged in at the time you request to use it. If you want to prompt the user, you need to do that yourself prior to beginning the SSH operation.

[zanda8893]

# Attempt SSH connection
log.debug(f"Connecting to {host.hostname} with user {host.user}")
if keys:
    for key in keys:
        log.debug(f"Using key: {key}")
        try:
            conn = await asyncssh.connect(
                host.hostname,
                port=host.port,
                username=host.user,
                password=host.password,
                client_keys=[key],
                known_hosts=None,
                tunnel=tunnel
            )
        except:
            continue
else:
    conn = await asyncssh.connect(
        host.hostname,
        port=host.port,
        username=host.user,
        password=host.password,
        client_keys=keys,
        known_hosts=None,
        tunnel=tunnel
    )

This workaround of looping though the keys and trying each in turn works but user interaction is required to check each key which is not ideal. Is there a better way?

[zanda8893]

Confirm user presence for key ECDSA-SK SHA256:gRPuqx63HwMVSgqnhWusKRMKAnQJJUdPc7icjeu+SCI
sign_and_send_pubkey: signing failed for ECDSA-SK "/home/zanda/.ssh/id_ecdsa_sk_rk_asgatewayorange": device not found
Confirm user presence for key ECDSA-SK SHA256:mo8hHpcQt/UbWJRZ0eGOX8Z/4LBxJd5duKfYdsU2Z9A
User presence confirmed

SSH can find the correct key without the need for user interaction. Not sure how they handle that.

[ronf]

I haven't done a lot of testing around this, but the expected behavior is that passing in client_keys=[key1, key2, key3, ...] should work with multiple of those keys being associated with security tokens. It will automatically skip over keys if the corresponding security key is not attached, but it should try all of the available keys until it finds one that is accepted by the remote server, just as it would if you specified multiple regular keys.

In your testing, is the correct key for the site plugged in before you invoke AsyncSSH?

[zanda8893]

The "Security key credential not found" error generally means that the security key you are trying to use is not currently attached to the system. If you have multiple security keys plugged in simultaneously, it should automatically try all of the available keys, generating hat error only when it runs out of keys to try (or if no keys were attached).

Keep in mind that AsyncSSH doesn't prompt for you to plug in a key. It expects it to be already plugged in at the time you request to use it. If you want to prompt the user, you need to do that yourself prior to beginning the SSH operation.

The code seems to error when trying the first key because it is not currently plugged in. If I catch the exception and continue to try a different private key if the security key is attached then it succeeds

[zanda8893]

I haven't done a lot of testing around this, but the expected behavior is that passing in client_keys=[key1, key2, key3, ...] should work with multiple of those keys being associated with security tokens. It will automatically skip over keys if the corresponding security key is not attached, but it should try all of the available keys until it finds one that is accepted by the remote server, just as it would if you specified multiple regular keys.

In your testing, is the correct key for the site plugged in before you invoke AsyncSSH?

Yes the correct key is attached and the second item in the keys list that I pass the connect function.

[ronf]

I just tested it here and it properly skipped over keys which were attached but had no corresponding entry in client_keys. I'll try it next with multiple keys being in client_keys.

[ronf]

Unfortunately, I only have one FIDO2 key here. My others are only capable of CTap1, and so there might be a difference in results depending on the key type.

When I add key1, key2 to my client_keys but only have the server trust key2 it works here, but it requires that I touch the first key to get it to move on to the second key, even though that first key isn't trusted by the server.

When I swap the order of the keys in client_keys, so that key2 is first in that list and trusted, it only requires me to touch key2, as expected, and it works fine with or without key1 attached to the system.

The only failure I've seen here is when trying to use an OpenSSH certificate with a CTap1 key. That can error out in a way that doesn't seem to try other keys.

[zanda8893]

Unfortunately, I only have one FIDO2 key here. My others are only capable of CTap1, and so there might be a difference in results depending on the key type.

When I add key1, key2 to my client_keys but only have the server trust key2 it works here, but it requires that I touch the first key to get it to move on to the second key, even though that first key isn't trusted by the server.

When I swap the order of the keys in client_keys, so that key2 is first in that list and trusted, it only requires me to touch key2, as expected, and it works fine with or without key1 attached to the system.

The only failure I've seen here is when trying to use an OpenSSH certificate with a CTap1 key. That can error out in a way that doesn't seem to try other keys.

I'm not sure about the behavior when multiple keys are attached. My main issue is when the server trusts both keys. Then when using the first I get the exception Security key credential not found rather then continuing to the next which would result in a successful connection.

[zanda8893]

I've looked a little further into this and the following script is able to extract the credential id and relaying party id from the private key file to compare with the key on the device without the need for user interaction. I am not very familiar with fido2 or how it works so this may not be suitable for the majority of security keys depending on how the SSH keys were generated. However for my use case it is able to identify the appropriate private key file for a attached fido2 security key.

from fido2.hid import CtapHidDevice
from fido2.ctap2 import Ctap2
import asyncssh

def extract_credential_id(filepath):
    # Load the private key file
    with open(filepath, 'rb') as f:
        key_data = f.read()

    # Import the private key from the loaded file
    key = asyncssh.import_private_key(key_data)

    # Extract the credential ID (key handle) from the key
    credential_id = key._key_handle

    # Print the credential ID in hexadecimal format for debugging purposes
    print(f"[DEBUG] Credential ID (hex): {credential_id.hex()}")
    print(key._comment)

    return credential_id, key._comment.decode('utf-8')

def check_credential_without_pin(credential_id, rp_id):
    # Find the FIDO device
    devices = list(CtapHidDevice.list_devices())
    if not devices:
        raise RuntimeError("No FIDO device found")

    device = devices[0]
    ctap2 = Ctap2(device)

    # Generate a client data hash from some challenge data
    challenge = b'dummychallenge'
    client_data_hash = sha256(challenge).digest()

    # Use get_assertion to check if the key recognizes the credential
    allow_list = [{"type": "public-key", "id": credential_id}]

    try:
        response = ctap2.get_assertion(
            rp_id=rp_id,  
            client_data_hash=client_data_hash,
            allow_list=allow_list,
            options={"up": False}  # Set user presence to false
        )
        print("[DEBUG] Credential found on the device.")
        return True
    except Exception as e:
        print(f"[DEBUG] Credential not found or error occurred: {e}")
        raise

# Example usage
# Extract the credential ID from the specified private key file
credential_id, rp_id = extract_credential_id(
    `[Path to id_ecdsa_sk_rk file]`'
)

# Check if the extracted credential ID matches any credential on the device without a PIN
check_credential_without_pin(credential_id, rp_id)

[ronf]

A few thoughts:

• This relies on accessing non-public members, like _key_handle, which could change in future releases.
• This currently only works for keys capable of talking Ctap2. Some older keys are still only capable of Ctap 1, like the original Google "Titan" security keys.
• There is a public get_comment() method you can use to avoid accessing _comment.
• Instead of using opening and reading from a file when importing a key, you can do this more directly with read_private_key().

It might not be too difficult to make a new method to query whether a security key is currently accessible or not. It would basically be a call to sk_sign(), but setting the option you did here to disable user presence. However, I'm still not sure why it correctly rotates between keys in some cases, while in others it fails. After additional testing I have seen some failures, where it returns the "Security key credential not found". I just need to figure out why that happens, and hopefully I can make the existing code just ignore keys which are loaded but don't have a corresponding security key attached.

[zanda8893]

I've updated the script with the changes you suggested. I'll be using that as a work around to find the correct key file before calling asyncssh.connect however it would be great to see this implemented in future. Thanks for you're help. I'm happy to do some more testing with this to try and figure out how exactly the exception is thrown.

[ronf]

Ok - I've got something available to test in commit 8593f28.

It turns out there were multiple issues in play here:

• There was a missing try..except in _ClientPublicKeyAuth which caught errors and called try_next_auth. This was present in _ClientHostBasedAuth, but missed in _ClientPublicKeyAuth.
• There was a problem with try_next_auth where it would always move on to the next auth method when it was called from anywhere other than process_userauth_failure. This would cause these cases to not test all of the client keys or client host keys if there was more than one in the list and a security key error was caught.
• Some security keys would enforce the touch requirement even when they didn't actually contain the key handle that was being tried. This could be confusing to users as they'd touch a key but still get an error back. The new code only sets touch_required after checking that the key contains the key_handle we're currently looking for.

With the new code, keys in client_keys which reference security keys which are not plugged into the system will be ignored. Also, signing will only be attempted if the server also trusts the key (which was already the case), so any key with a touch requirement will only have that triggered when the key is referenced in client_keys, plugged into the client machine, and trusted by the server.

I did testing here with various combinations of keys being plugged into the system and varied whether they were present in client_keys and/or authorized_keys and things are looking good to me. Let me know how you make out with it!