different hostkey and algorithm compare with ssh client.

ericwq commented 9 months ago

base known_hosts

here is the base ~/.ssh/known_hosts file.

qiwang@Qi15Pro .ssh % more known_hosts
localhost ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOoLveVRGZHdwPX70TZxScl0hgf94gSF+HaM/RMlIAGB
localhost ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC5UUsy6H8B/RzOtvNOuDyzaRta4PquwUtKH9JeWkqg9oAW+esAE98vEACjZ3pIqhFC9QjrRVjB6cHk2j3Q4gCVCkiyZOvPwMPwUfq/M97hZq1nwJre7V+2245ls/3mdnL/6dJu3GNckqueyBImKhBAz8gNDzqsKGshzcWwHW523Ktd+QZqsqtfhEB4C09wqcRN+BrNwkmPItfshOGd4AYwmZUGADxUhcg1MG0TakakNLSj6jL7aKPEbm7dYVj/F/TQQGapmA/p++xeodUmwcpGCaXZml3nKIjkqWCgZIGECkQKN6Cm+yGvFSgENRf/2n4qxGp2Vy6eMYRDc5LluFN3rdc6bCOBOM0NIi6IkkHoxVkxeSxOPt7UFlfaVEm1J5BmmkSDDyHoYpo+nNzqFfrnDPW7JBc8GIwbiKtH++EMiHTU+0c5YNXdOtiHxWaLTvUTJobzyMdsbQ8Ahy12ABYpUUJgbLPae9IVepwA49WMrPzM+8GJCncOvwxLpW02XNs=
localhost ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFmkUUaDuZhC3T95dus3F+W7wuhKhc55RCTtqb0DZOX9jg/GB7pG7SZKMkldZpkQd2BPYSFGg624/mgDD5WBCYU=

output from ssh client

Then I login with ssh -p 8022 ide@localhost:

qiwang@Qi15Pro client % ssh -p 8022 ide@localhost
The authenticity of host '[localhost]:8022 ([::1]:8022)' can't be established.
ED25519 key fingerprint is SHA256:O70nOfpgMg+4JRdyiRRX1DUXrHf8BNmbTymbzwVaXCA.
This key is not known by any other names
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '[localhost]:8022' (ED25519) to the list of known hosts.
Welcome to Alpine!

The Alpine Wiki contains a large amount of how-to guides and general
information about administrating Alpine systems.
See <https://wiki.alpinelinux.org/>.

You can setup the system with the command: setup-alpine

You may change this message by editing /etc/motd.

openrc-abuild:~$

known_hosts changed by ssh client

here is the ~/.ssh/known_hosts file after ssh client login. we notice that 3 lines of [localhost]:8022 was added.

qiwang@Qi15Pro .ssh % more known_hosts
localhost ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOoLveVRGZHdwPX70TZxScl0hgf94gSF+HaM/RMlIAGB
localhost ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC5UUsy6H8B/RzOtvNOuDyzaRta4PquwUtKH9JeWkqg9oAW+esAE98vEACjZ3pIqhFC9QjrRVjB6cHk2j3Q4gCVCkiyZOvPwMPwUfq/M97hZq1nwJre7V+2245ls/3mdnL/6dJu3GNckqueyBImKhBAz8gNDzqsKGshzcWwHW523Ktd+QZqsqtfhEB4C09wqcRN+BrNwkmPItfshOGd4AYwmZUGADxUhcg1MG0TakakNLSj6jL7aKPEbm7dYVj/F/TQQGapmA/p++xeodUmwcpGCaXZml3nKIjkqWCgZIGECkQKN6Cm+yGvFSgENRf/2n4qxGp2Vy6eMYRDc5LluFN3rdc6bCOBOM0NIi6IkkHoxVkxeSxOPt7UFlfaVEm1J5BmmkSDDyHoYpo+nNzqFfrnDPW7JBc8GIwbiKtH++EMiHTU+0c5YNXdOtiHxWaLTvUTJobzyMdsbQ8Ahy12ABYpUUJgbLPae9IVepwA49WMrPzM+8GJCncOvwxLpW02XNs=
localhost ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFmkUUaDuZhC3T95dus3F+W7wuhKhc55RCTtqb0DZOX9jg/GB7pG7SZKMkldZpkQd2BPYSFGg624/mgDD5WBCYU=
[localhost]:8022 ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAILLprakLbWrM0uWPS2ToU1JvDW+B/Of9kIxqBD/E6uTE
[localhost]:8022 ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDTARZ2ZBYlS4zfE/yTC/clj6xeMQXwtmuw5XDdzB+MBFBOzqVeryvDZd7iTmlb36xuPsz4Fy+gg+2aHbkl0EdFWolSKIvnTwOo8N7okPWF/x8CGcPE9lslVd7W0HMsWay6S+uS93hlIIrHESgl1cTxi1yJRdJ4GoTnTJ+9LGzyQWKlzPUychP/vv2OxjVoLt/jgeVqbg43RwFSaI8WWUbU6gD/1Uu0pRJTaPBWDsF+1Gx1V4Go6HgSl9MemVhZKFmVHQIxU+l9TuuUA5lJg5bZ4OHaxzI0Tc9FiEdaNIN3hRA+1ZpPpc4Ig7seASKQ4AVRRZ8uDX9tjqlejy6z/3FNbdcypZHoQaMhWqVeM1wctvSo05OuoqyojF9zzWJiHhkaxTwi0NPvjilye0Vz4z6kcyOmra2dMvakBgBgFpzRtCceRNGQDonclX4WxDuMsS+VcmzfzCrICPJRLwgnvHGSKLn4rucN0VJpzsY1EX4yv0xD2oZj8HtSgndpRlwKYec=
[localhost]:8022 ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCr9GJXG1e4qyz3Oh9Z2xuYiLSxVj8SQs6kOTcvLCQPGwZlAG27sXc3/HXIzYvB7WPcUKqNdxcFndfWhPsr/sEM=

my application

then I restore ~/.ssh/known_hosts file to remove the lines added by ssh client. finally, I use the following code to call my application (as bellow).

    sshHost := c.host + ":" + c.sshPort
    khPath := filepath.Join(os.Getenv("HOME"), ".ssh", "known_hosts")
    kh, err := knownhosts.New(khPath)
    if err != nil {
        return err
    }

    // Create a custom permissive hostkey callback which still errors on hosts
    // with changed keys, but allows unknown hosts and adds them to known_hosts
    cb := ssh.HostKeyCallback(func(hostname string, remote net.Addr, key ssh.PublicKey) error {
        err := kh(hostname, remote, key)
        if knownhosts.IsHostKeyChanged(err) {
            return fmt.Errorf("REMOTE HOST IDENTIFICATION HAS CHANGED for host %s! This may indicate a MitM attack.", hostname)
        } else if knownhosts.IsHostUnknown(err) {

            hint := "The authenticity of host '%s (%s)' can't be established.\n" +
                "%s key fingerprint is %s.\n" +
                "This key is not known by any other names\n" +
                "Are you sure you want to continue connecting (yes/no/[fingerprint])?"
            fmt.Printf(hint, hostname, remote, strings.ToUpper(key.Type()), ssh.FingerprintSHA256(key))

            var answer string
            fmt.Scanln(&answer)
            switch answer {
            case "yes", "y":
                fmt.Printf("Warning: Permanently added '%s' (%s) to the list of known hosts.\n",
                    hostname, strings.ToUpper(key.Type()))

                f, ferr := os.OpenFile(khPath, os.O_APPEND|os.O_WRONLY, 0600)
                if ferr == nil {
                    defer f.Close()
                    ferr = knownhosts.WriteKnownHost(f, hostname, remote, key)
                }
                if ferr == nil {
                    fmt.Printf("Added host %s to known_hosts\n", hostname)
                } else {
                    fmt.Printf("Failed to add host %s to known_hosts: %v\n", hostname, ferr)
                }
                return nil // permit previously-unknown hosts (warning: may be insecure)
            case "no", "n":
                fmt.Println("Host key verification failed.")
                return err
            }
        }
        return err
    })

    clientConfig := &ssh.ClientConfig{
        User:              c.user,
        Auth:              auth,
        HostKeyCallback:   cb,
        HostKeyAlgorithms: kh.HostKeyAlgorithms(sshHost),
        Timeout:           time.Duration(3) * time.Second,
    }

    // TODO understand ssh login session, is that possible to replace the sshd depdends?
    client, err := ssh.Dial("tcp", sshHost, clientConfig)
    if err != nil {
        return err
    }
    defer client.Close()

output of my application

here is the output of my application.

qiwang@Qi15Pro client % GOCOVERDIR=./coverage/int  ~/.local/bin/apsh  ide@localhost:8022
The authenticity of host 'localhost:8022 ([::1]:8022)' can't be established.
ECDSA-SHA2-NISTP256 key fingerprint is SHA256:7kG09hr4PefUaFvjT+O3LAPWHPY9CcAxu/eiaYawBRM.
This key is not known by any other names
Are you sure you want to continue connecting (yes/no/[fingerprint])?yes
Warning: Permanently added 'localhost:8022' (ECDSA-SHA2-NISTP256) to the list of known hosts.
Added host localhost:8022 to known_hosts
Process exited with status 127
qiwang@Qi15Pro client %

known_hosts changed by my application

here is the ~/.ssh/known_hosts file after run my application. There is only one line (ecdsa-sha2-nistp256) was added.

more known_hosts
localhost ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOoLveVRGZHdwPX70TZxScl0hgf94gSF+HaM/RMlIAGB
localhost ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC5UUsy6H8B/RzOtvNOuDyzaRta4PquwUtKH9JeWkqg9oAW+esAE98vEACjZ3pIqhFC9QjrRVjB6cHk2j3Q4gCVCkiyZOvPwMPwUfq/M97hZq1nwJre7V+2245ls/3mdnL/6dJu3GNckqueyBImKhBAz8gNDzqsKGshzcWwHW523Ktd+QZqsqtfhEB4C09wqcRN+BrNwkmPItfshOGd4AYwmZUGADxUhcg1MG0TakakNLSj6jL7aKPEbm7dYVj/F/TQQGapmA/p++xeodUmwcpGCaXZml3nKIjkqWCgZIGECkQKN6Cm+yGvFSgENRf/2n4qxGp2Vy6eMYRDc5LluFN3rdc6bCOBOM0NIi6IkkHoxVkxeSxOPt7UFlfaVEm1J5BmmkSDDyHoYpo+nNzqFfrnDPW7JBc8GIwbiKtH++EMiHTU+0c5YNXdOtiHxWaLTvUTJobzyMdsbQ8Ahy12ABYpUUJgbLPae9IVepwA49WMrPzM+8GJCncOvwxLpW02XNs=
localhost ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFmkUUaDuZhC3T95dus3F+W7wuhKhc55RCTtqb0DZOX9jg/GB7pG7SZKMkldZpkQd2BPYSFGg624/mgDD5WBCYU=
[localhost]:8022,[::1]:8022 ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCr9GJXG1e4qyz3Oh9Z2xuYiLSxVj8SQs6kOTcvLCQPGwZlAG27sXc3/HXIzYvB7WPcUKqNdxcFndfWhPsr/sEM=

compare with ssh client

key return to client

ssh client and my application return different keys for user to choose. different key has different fingerprint.

ssh client return ED25519 key,
my application return ECDSA-SHA2-NISTP256 key.

hostkeys in known_host

ssh client add 3 host keys to ~/.ssh/known_hosts file. it's ssh-ed25519, ssh-rsa, ecdsa-sha2-nistp256.
my application add 1 host key to ~/.ssh/known_hosts file, it's ecdsa-sha2-nistp256

@evanelias @lonnywong do you have any idea/suggestion about the compare result?

ericwq commented 9 months ago

after see this x/crypto/ssh: add support for hostkeys@openssh.com, I don't know if this is the reason: x/crypto/ssh doesn't support hostkeys@openssh.com yet.

evanelias commented 9 months ago

I believe this is due to the behavior of golang.org/x/crypto/ssh -- the ClientConfig.HostKeyCallback will only be called once, even if the server has multiple public keys. The ClientConfig.HostKeyAlgorithms specifies the list of acceptable algorithms, in order of preference. When a server has multiple acceptable public keys, only one will be chosen, deterministically based on that ordering. This single public key is then passed to the HostKeyCallback, which either accepts (nil return) or rejects (non-nil error return) this server key, but either way it does not receive any other server keys or get called multiple times.

github.com/skeema/knownhosts provides a benefit in a different but slightly related scenario: server has multiple public keys, and some (but not all) of those keys are already in the knownhosts file. In this case github.com/skeema/knownhosts is useful for easily populating ClientConfig.HostKeyAlgorithms based on the already-known keys from the knownhosts file.

However, your situation here is different, if I understand correctly: server has multiple public keys, none of which are in the knownhosts file yet, and you wish to add all of them to the knownhosts file. github.com/skeema/knownhosts cannot directly help with this situation, because it doesn't control how or when the HostKeyCallback function is invoked by logic in golang.org/x/crypto/ssh.

I don't know if there's any way in golang.org/x/crypto/ssh to obtain the server's full list of public keys from a single client connection. In theory you could probe a server by making a bunch of different client connections, shrinking the HostKeyAlgorithms list each time, but this is hacky and grossly inefficient.

Sorry that github.com/skeema/knownhosts can't solve this directly. If you do find a good solution, please comment back here, I'd be curious to know too. Thanks!

ericwq commented 8 months ago

thanks for your patient reply. it helps me a lot. I will pay attention to this problem. will notify you if it make any progress.

skeema / knownhosts