Troubleshooting steps https://github.com/PowerShell/Win32-OpenSSH/wiki/Troubleshooting-Steps

Terminal issue? please go through wiki https://github.com/PowerShell/Win32-OpenSSH/wiki/TTY-PTY-support-in-Windows-OpenSSH

Please answer the following

"OpenSSH for Windows" version OpenSSH_for_Windows_8.1p1, LibreSSL 2.9.2

Server OperatingSystem Ubuntu 20.04.3 LTS OpenSSH_8.2p1 Ubuntu-4ubuntu0.3, OpenSSL 1.1.1f

Client OperatingSystem Windows Server 2019 Standard OpenSSH_for_Windows_8.1p1, LibreSSL 2.9.2

What is failing The corporate environment is running on a Windows Domain Controller and we are trying to get Windows Client computer to SSH to Linux server to execute linux programs remotely. Both client and servers are domain joined. Ubuntu server runs sssd and PAM for authentication domain accounts and the following GSSAPI options are turned on:

GSSAPIAuthentication yes GSSAPIKeyExchange yes

What we are trying to achieve with GSSAPI auth is SSO through pam stack for passwordless automation and file share mount (pam_mount). Pub-Key auth works but it does not integrate with sssd so PAM modules are triggered without AD credential, therefore fail to mount the SMB file share. We are hoping that GSSAPI can provide passwordless auth while passing keytab to the server for mounting (via mount.cifs -o sec=krb5)

However, ssh seems to attempt authentication via gssapi but without success. Since it is using Microsoft SSPI, I don't have a way to dig further into the what went wrong.

On the AD side, we didn't turn on the encryption for the service account we are using to connect since it isn't on by default. On the server machine we also didn't turn on "Trust this computer for delegation" option. Would you recommend to retry with these two options turned on?

Expected output for client logs: debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password debug3: start over, passed a different list publickey,gssapi-keyex,gssapi-with-mic,password debug3: preferred gssapi-with-mic,publickey,keyboard-interactive,password debug3: authmethod_lookup gssapi-with-mic debug3: remaining preferred: publickey,keyboard-interactive,password debug3: authmethod_is_enabled gssapi-with-mic debug1: Next authentication method: gssapi-with-mic debug3: send packet: type 50 debug2: we sent a gssapi-with-mic packet, wait for reply debug3: receive packet: type 60 debug1: Delegating credentials debug1: sspi delegation was requested but not fulfilled debug3: send packet: type 61 debug3: receive packet: type 61 debug1: Delegating credentials debug1: sspi delegation was requested but not fulfilled debug3: send packet: type 66 debug3: receive packet: type 52 debug1: Authentication succeeded (gssapi-with-mic).

Actual output debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password debug3: start over, passed a different list publickey,gssapi-keyex,gssapi-with-mic,password debug3: preferred gssapi-with-mic,publickey,keyboard-interactive,password debug3: authmethod_lookup gssapi-with-mic debug3: remaining preferred: publickey,keyboard-interactive,password debug3: authmethod_is_enabled gssapi-with-mic debug1: Next authentication method: gssapi-with-mic debug3: send packet: type 50 debug2: we sent a gssapi-with-mic packet, wait for reply debug3: receive packet: type 60 debug1: Delegating credentials debug1: sspi delegation was requested but not fulfilled debug3: send packet: type 61 debug3: receive packet: type 61 debug1: Delegating credentials debug1: sspi delegation was requested but not fulfilled debug3: send packet: type 66 debug3: receive packet: type 51

Full client and server logs will be provided below

Client log (windows):

OpenSSH_for_Windows_8.1p1, LibreSSL 2.9.2 debug3: Failed to open file:C:/Users/testuser/.ssh/config error:2 debug3: Failed to open file:C:/ProgramData/ssh/ssh_config error:2 debug2: resolving "clientdns" port 2222 debug2: ssh_connect_direct debug1: Connecting to clientdns [10.1.27.48] port 2222. debug1: Connection established. debug3: Failed to open file:C:/Users/testuser/.ssh/id_rsa error:2 debug3: Failed to open file:C:/Users/testuser/.ssh/id_rsa.pub error:2 debug1: identity file C:\Users\testuser/.ssh/id_rsa type -1 debug3: Failed to open file:C:/Users/testuser/.ssh/id_rsa-cert error:2 debug3: Failed to open file:C:/Users/testuser/.ssh/id_rsa-cert.pub error:2 debug1: identity file C:\Users\testuser/.ssh/id_rsa-cert type -1 debug3: Failed to open file:C:/Users/testuser/.ssh/id_dsa error:2 debug3: Failed to open file:C:/Users/testuser/.ssh/id_dsa.pub error:2 debug1: identity file C:\Users\testuser/.ssh/id_dsa type -1 debug3: Failed to open file:C:/Users/testuser/.ssh/id_dsa-cert error:2 debug3: Failed to open file:C:/Users/testuser/.ssh/id_dsa-cert.pub error:2 debug1: identity file C:\Users\testuser/.ssh/id_dsa-cert type -1 debug3: Failed to open file:C:/Users/testuser/.ssh/id_ecdsa error:2 debug3: Failed to open file:C:/Users/testuser/.ssh/id_ecdsa.pub error:2 debug1: identity file C:\Users\testuser/.ssh/id_ecdsa type -1 debug3: Failed to open file:C:/Users/testuser/.ssh/id_ecdsa-cert error:2 debug3: Failed to open file:C:/Users/testuser/.ssh/id_ecdsa-cert.pub error:2 debug1: identity file C:\Users\testuser/.ssh/id_ecdsa-cert type -1 debug3: Failed to open file:C:/Users/testuser/.ssh/id_ed25519 error:2 debug3: Failed to open file:C:/Users/testuser/.ssh/id_ed25519.pub error:2 debug1: identity file C:\Users\testuser/.ssh/id_ed25519 type -1 debug3: Failed to open file:C:/Users/testuser/.ssh/id_ed25519-cert error:2 debug3: Failed to open file:C:/Users/testuser/.ssh/id_ed25519-cert.pub error:2 debug1: identity file C:\Users\testuser/.ssh/id_ed25519-cert type -1 debug3: Failed to open file:C:/Users/testuser/.ssh/id_xmss error:2 debug3: Failed to open file:C:/Users/testuser/.ssh/id_xmss.pub error:2 debug1: identity file C:\Users\testuser/.ssh/id_xmss type -1 debug3: Failed to open file:C:/Users/testuser/.ssh/id_xmss-cert error:2 debug3: Failed to open file:C:/Users/testuser/.ssh/id_xmss-cert.pub error:2 debug1: identity file C:\Users\testuser/.ssh/id_xmss-cert type -1 debug1: Local version string SSH-2.0-OpenSSH_for_Windows_8.1 debug1: Remote protocol version 2.0, remote software version OpenSSH_8.2p1 Ubuntu-4ubuntu0.3 debug1: match: OpenSSH_8.2p1 Ubuntu-4ubuntu0.3 pat OpenSSH* compat 0x04000000 debug2: fd 3 setting O_NONBLOCK debug1: Authenticating to clientdns:2222 as 'domain\testuser' debug3: put_host_port: [clientdns]:2222 debug3: hostkeys_foreach: reading file "C:\Users\testuser/.ssh/known_hosts" debug3: Failed to open file:C:/Users/testuser/.ssh/known_hosts2 error:2 debug3: Failed to open file:C:/ProgramData/ssh/ssh_known_hosts error:2 debug3: Failed to open file:C:/ProgramData/ssh/ssh_known_hosts2 error:2 debug3: send packet: type 20 debug1: SSH2_MSG_KEXINIT sent debug3: receive packet: type 20 debug1: SSH2_MSG_KEXINIT received debug2: local client KEXINIT proposal debug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,ext-info-c debug2: host key algorithms: ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ssh-ed25519-cert-v01@openssh.com,rsa-sha2-512-cert-v01@openssh.com,rsa-sha2-256-cert-v01@openssh.com,ssh-rsa-cert-v01@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-ed25519,rsa-sha2-512,rsa-sha2-256,ssh-rsa debug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com debug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com debug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1 debug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1 debug2: compression ctos: none,zlib@openssh.com,zlib debug2: compression stoc: none,zlib@openssh.com,zlib debug2: languages ctos: debug2: languages stoc: debug2: first_kex_follows 0 debug2: reserved 0 debug2: peer server KEXINIT proposal debug2: KEX algorithms: gss-gex-sha1-toWM5Slw5Ew8Mqkay+al2g==,gss-group14-sha1-toWM5Slw5Ew8Mqkay+al2g==,curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256 debug2: host key algorithms: rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519 debug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com debug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com debug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1 debug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1 debug2: compression ctos: none,zlib@openssh.com debug2: compression stoc: none,zlib@openssh.com debug2: languages ctos: debug2: languages stoc: debug2: first_kex_follows 0 debug2: reserved 0 debug1: kex: algorithm: curve25519-sha256 debug1: kex: host key algorithm: ecdsa-sha2-nistp256 debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: compression: none debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: compression: none debug3: send packet: type 30 debug1: expecting SSH2_MSG_KEX_ECDH_REPLY debug3: receive packet: type 31 debug1: Server host key: ecdsa-sha2-nistp256 SHA256:Tk6LVFbMXjnIJK1sP+9CWHcIzsOuDeeSDoKwPopmAmw debug3: put_host_port: [10.1.27.48]:2222 debug3: put_host_port: [clientdns]:2222 debug3: hostkeys_foreach: reading file "C:\Users\testuser/.ssh/known_hosts" debug3: Failed to open file:C:/Users/testuser/.ssh/known_hosts2 error:2 debug3: Failed to open file:C:/ProgramData/ssh/ssh_known_hosts error:2 debug3: Failed to open file:C:/ProgramData/ssh/ssh_known_hosts2 error:2 debug3: hostkeys_foreach: reading file "C:\Users\testuser/.ssh/known_hosts" debug3: Failed to open file:C:/Users/testuser/.ssh/known_hosts2 error:2 debug3: Failed to open file:C:/ProgramData/ssh/ssh_known_hosts error:2 debug3: Failed to open file:C:/ProgramData/ssh/ssh_known_hosts2 error:2 debug1: checking without port identifier debug3: hostkeys_foreach: reading file "C:\Users\testuser/.ssh/known_hosts" debug3: record_hostkey: found key type ECDSA in file C:\Users\testuser/.ssh/known_hosts:1 debug3: load_hostkeys: loaded 1 keys from clientdns debug3: Failed to open file:C:/Users/testuser/.ssh/known_hosts2 error:2 debug3: Failed to open file:C:/ProgramData/ssh/ssh_known_hosts error:2 debug3: Failed to open file:C:/ProgramData/ssh/ssh_known_hosts2 error:2 debug3: hostkeys_foreach: reading file "C:\Users\testuser/.ssh/known_hosts" debug3: record_hostkey: found key type ECDSA in file C:\Users\testuser/.ssh/known_hosts:1 debug3: load_hostkeys: loaded 1 keys from 10.1.27.48 debug3: Failed to open file:C:/Users/testuser/.ssh/known_hosts2 error:2 debug3: Failed to open file:C:/ProgramData/ssh/ssh_known_hosts error:2 debug3: Failed to open file:C:/ProgramData/ssh/ssh_known_hosts2 error:2 debug1: Host 'clientdns' is known and matches the ECDSA host key. debug1: Found key in C:\Users\testuser/.ssh/known_hosts:1 debug1: found matching key w/out port debug3: send packet: type 21 debug2: set_newkeys: mode 1 debug1: rekey out after 134217728 blocks debug1: SSH2_MSG_NEWKEYS sent debug1: expecting SSH2_MSG_NEWKEYS debug3: receive packet: type 21 debug1: SSH2_MSG_NEWKEYS received debug2: set_newkeys: mode 0 debug1: rekey in after 134217728 blocks debug3: unable to connect to pipe \\.\pipe\openssh-ssh-agent, error: 2 debug1: pubkey_prepare: ssh_get_authentication_socket: No such file or directory debug1: Will attempt key: C:\Users\testuser/.ssh/id_rsa debug1: Will attempt key: C:\Users\testuser/.ssh/id_dsa debug1: Will attempt key: C:\Users\testuser/.ssh/id_ecdsa debug1: Will attempt key: C:\Users\testuser/.ssh/id_ed25519 debug1: Will attempt key: C:\Users\testuser/.ssh/id_xmss debug2: pubkey_prepare: done debug3: send packet: type 5 debug3: receive packet: type 7 debug1: SSH2_MSG_EXT_INFO received debug1: kex_input_ext_info: server-sig-algs=ssh-ed25519,sk-ssh-ed25519@openssh.com,ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ecdsa-sha2-nistp256@openssh.com debug3: receive packet: type 6 debug2: service_accept: ssh-userauth debug1: SSH2_MSG_SERVICE_ACCEPT received debug3: send packet: type 50 debug3: receive packet: type 53 debug3: input_userauth_banner Authorized uses only. All activities performed on this system will be monitored. debug3: receive packet: type 51 debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password debug3: start over, passed a different list publickey,gssapi-keyex,gssapi-with-mic,password debug3: preferred gssapi-with-mic,publickey,keyboard-interactive,password debug3: authmethod_lookup gssapi-with-mic debug3: remaining preferred: publickey,keyboard-interactive,password debug3: authmethod_is_enabled gssapi-with-mic debug1: Next authentication method: gssapi-with-mic debug3: send packet: type 50 debug2: we sent a gssapi-with-mic packet, wait for reply debug3: receive packet: type 60 debug1: Delegating credentials debug1: sspi delegation was requested but not fulfilled debug3: send packet: type 61 debug3: receive packet: type 61 debug1: Delegating credentials debug1: sspi delegation was requested but not fulfilled debug3: send packet: type 66 debug3: receive packet: type 51 debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password debug2: we did not send a packet, disable method debug3: authmethod_lookup publickey debug3: remaining preferred: keyboard-interactive,password debug3: authmethod_is_enabled publickey debug1: Next authentication method: publickey debug1: Trying private key: C:\Users\testuser/.ssh/id_rsa debug3: no such identity: C:\Users\testuser/.ssh/id_rsa: No such file or directory debug1: Trying private key: C:\Users\testuser/.ssh/id_dsa debug3: no such identity: C:\Users\testuser/.ssh/id_dsa: No such file or directory debug1: Trying private key: C:\Users\testuser/.ssh/id_ecdsa debug3: no such identity: C:\Users\testuser/.ssh/id_ecdsa: No such file or directory debug1: Trying private key: C:\Users\testuser/.ssh/id_ed25519 debug3: no such identity: C:\Users\testuser/.ssh/id_ed25519: No such file or directory debug1: Trying private key: C:\Users\testuser/.ssh/id_xmss debug3: no such identity: C:\Users\testuser/.ssh/id_xmss: No such file or directory debug2: we did not send a packet, disable method debug3: authmethod_lookup password debug3: remaining preferred: ,password debug3: authmethod_is_enabled password debug1: Next authentication method: password debug3: failed to open file:C:/dev/tty error:3 debug1: read_passphrase: can't open /dev/tty: No such file or directory domain\testuser@clientdns's password:

Server Log:

debug2: load_server_config: filename /etc/ssh/sshd_config debug2: load_server_config: done config len = 626 debug2: parse_server_config_depth: config /etc/ssh/sshd_config len 626 debug2: /etc/ssh/sshd_config line 13: new include /etc/ssh/sshd_config.d/.conf debug2: /etc/ssh/sshd_config line 13: including /etc/ssh/sshd_config.d/DOMAIN_Auth_Key.conf debug2: load_server_config: filename /etc/ssh/sshd_config.d/DOMAIN_Auth_Key.conf debug2: load_server_config: done config len = 106 debug2: parse_server_config_depth: config /etc/ssh/sshd_config.d/DOMAIN_Auth_Key.conf len 106 debug3: /etc/ssh/sshd_config.d/DOMAIN_Auth_Key.conf:1 setting AuthorizedKeysFile /etc/ssh/keys/%u/authorized_keys debug3: checking syntax for 'Match User svcQremote' debug3: /etc/ssh/sshd_config:29 setting LogLevel INFO debug3: /etc/ssh/sshd_config:34 setting LoginGraceTime 60 debug3: /etc/ssh/sshd_config:36 setting PermitRootLogin no debug3: /etc/ssh/sshd_config:39 setting MaxAuthTries 4 debug3: /etc/ssh/sshd_config:40 setting MaxSessions 10 debug3: /etc/ssh/sshd_config:54 setting HostbasedAuthentication no debug3: /etc/ssh/sshd_config:59 setting IgnoreRhosts yes debug3: /etc/ssh/sshd_config:63 setting PermitEmptyPasswords no debug3: /etc/ssh/sshd_config:67 setting ChallengeResponseAuthentication no debug3: /etc/ssh/sshd_config:76 setting GSSAPIAuthentication yes debug3: /etc/ssh/sshd_config:79 setting GSSAPIKeyExchange yes debug3: /etc/ssh/sshd_config:90 setting UsePAM yes debug3: /etc/ssh/sshd_config:96 setting X11Forwarding no debug3: /etc/ssh/sshd_config:100 setting PrintMotd no debug3: /etc/ssh/sshd_config:103 setting PermitUserEnvironment no debug3: /etc/ssh/sshd_config:106 setting ClientAliveInterval 300 debug3: /etc/ssh/sshd_config:107 setting ClientAliveCountMax 3 debug3: /etc/ssh/sshd_config:111 setting MaxStartups 10:30:60 debug3: /etc/ssh/sshd_config:118 setting Banner /etc/issue.net debug3: /etc/ssh/sshdconfig:121 setting AcceptEnv LANG LC debug3: /etc/ssh/sshd_config:124 setting Subsystem sftp /usr/lib/openssh/sftp-server debug3: /etc/ssh/sshd_config:132 setting PasswordAuthentication yes debug1: sshd version OpenSSH_8.2, OpenSSL 1.1.1f 31 Mar 2020 debug1: private host key #0: ssh-rsa SHA256:X4Nfc26X9hn15z+gWtrvqn0G54o4etTEl9gKcW/f6hI debug1: private host key #1: ecdsa-sha2-nistp256 SHA256:Tk6LVFbMXjnIJK1sP+9CWHcIzsOuDeeSDoKwPopmAmw debug1: private host key #2: ssh-ed25519 SHA256:6bqsDeIw6RSNeZ0JIU3QHZpLzDqz8lUIvwCzeiLXJTU debug1: rexec_argv[0]='/usr/sbin/sshd' debug1: rexec_argv[1]='-ddd' debug1: rexec_argv[2]='-p' debug1: rexec_argv[3]='2222' debug3: oom_adjust_setup debug1: Set /proc/self/oom_score_adj from 0 to -1000 debug2: fd 3 setting O_NONBLOCK debug1: Bind to port 2222 on 0.0.0.0. Server listening on 0.0.0.0 port 2222. debug2: fd 4 setting O_NONBLOCK debug3: sock_set_v6only: set socket 4 IPV6_V6ONLY debug1: Bind to port 2222 on ::. Server listening on :: port 2222. debug3: fd 5 is not O_NONBLOCK debug1: Server will not fork when running in debugging mode. debug3: send_rexec_state: entering fd = 8 config len 626 debug3: ssh_msg_send: type 0 debug3: send_rexec_state: done debug1: rexec start in 5 out 5 newsock 5 pipe -1 sock 8 debug3: recv_rexec_state: entering fd = 5 debug3: ssh_msg_recv entering debug3: recv_rexec_state: done debug2: parse_server_config_depth: config rexec len 626 debug2: parse_server_config_depth: config /etc/ssh/sshd_config.d/DOMAIN_Auth_Key.conf len 106 debug3: /etc/ssh/sshd_config.d/DOMAIN_Auth_Key.conf:1 setting AuthorizedKeysFile /etc/ssh/keys/%u/authorizedkeys debug3: checking syntax for 'Match User svcQremote' debug3: rexec:29 setting LogLevel INFO debug3: rexec:34 setting LoginGraceTime 60 debug3: rexec:36 setting PermitRootLogin no debug3: rexec:39 setting MaxAuthTries 4 debug3: rexec:40 setting MaxSessions 10 debug3: rexec:54 setting HostbasedAuthentication no debug3: rexec:59 setting IgnoreRhosts yes debug3: rexec:63 setting PermitEmptyPasswords no debug3: rexec:67 setting ChallengeResponseAuthentication no debug3: rexec:76 setting GSSAPIAuthentication yes debug3: rexec:79 setting GSSAPIKeyExchange yes debug3: rexec:90 setting UsePAM yes debug3: rexec:96 setting X11Forwarding no debug3: rexec:100 setting PrintMotd no debug3: rexec:103 setting PermitUserEnvironment no debug3: rexec:106 setting ClientAliveInterval 300 debug3: rexec:107 setting ClientAliveCountMax 3 debug3: rexec:111 setting MaxStartups 10:30:60 debug3: rexec:118 setting Banner /etc/issue.net debug3: rexec:121 setting AcceptEnv LANG LC debug3: rexec:124 setting Subsystem sftp /usr/lib/openssh/sftp-server debug3: rexec:132 setting PasswordAuthentication yes debug1: sshd version OpenSSH_8.2, OpenSSL 1.1.1f 31 Mar 2020 debug1: private host key #0: ssh-rsa SHA256:X4Nfc26X9hn15z+gWtrvqn0G54o4etTEl9gKcW/f6hI debug1: private host key #1: ecdsa-sha2-nistp256 SHA256:Tk6LVFbMXjnIJK1sP+9CWHcIzsOuDeeSDoKwPopmAmw debug1: private host key #2: ssh-ed25519 SHA256:6bqsDeIw6RSNeZ0JIU3QHZpLzDqz8lUIvwCzeiLXJTU debug1: inetd sockets after dupping: 3, 3 Connection from 10.1.35.100 port 20520 on 10.1.27.48 port 2222 rdomain "" debug1: Local version string SSH-2.0-OpenSSH_8.2p1 Ubuntu-4ubuntu0.3 debug1: Remote protocol version 2.0, remote software version OpenSSH_for_Windows_8.1 debug1: match: OpenSSH_for_Windows_8.1 pat OpenSSH compat 0x04000000 debug2: fd 3 setting O_NONBLOCK debug3: ssh_sandbox_init: preparing seccomp filter sandbox debug2: Network child is on pid 82176 debug3: preauth child monitor started debug3: privsep user:group 112:65534 [preauth] debug1: permanently_set_uid: 112/65534 [preauth] debug3: ssh_sandbox_child: setting PR_SET_NO_NEW_PRIVS [preauth] debug3: ssh_sandbox_child: attaching seccomp filter program [preauth] debug1: list_hostkey_types: rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519 [preauth] debug3: mm_request_send entering: type 42 [preauth] debug3: mm_request_receive_expect entering: type 43 [preauth] debug3: mm_request_receive entering [preauth] debug3: mm_request_receive entering debug3: monitor_read: checking request 42 debug3: mm_request_send entering: type 43 debug3: send packet: type 20 [preauth] debug1: SSH2_MSG_KEXINIT sent [preauth] debug3: receive packet: type 20 [preauth] debug1: SSH2_MSG_KEXINIT received [preauth] debug2: local server KEXINIT proposal [preauth] debug2: KEX algorithms: gss-gex-sha1-toWM5Slw5Ew8Mqkay+al2g==,gss-group14-sha1-toWM5Slw5Ew8Mqkay+al2g==,curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256 [preauth] debug2: host key algorithms: rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519 [preauth] debug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com [preauth] debug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com [preauth] debug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1 [preauth] debug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1 [preauth] debug2: compression ctos: none,zlib@openssh.com [preauth] debug2: compression stoc: none,zlib@openssh.com [preauth] debug2: languages ctos: [preauth] debug2: languages stoc: [preauth] debug2: first_kex_follows 0 [preauth] debug2: reserved 0 [preauth] debug2: peer client KEXINIT proposal [preauth] debug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,ext-info-c [preauth] debug2: host key algorithms: ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ssh-ed25519-cert-v01@openssh.com,rsa-sha2-512-cert-v01@openssh.com,rsa-sha2-256-cert-v01@openssh.com,ssh-rsa-cert-v01@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-ed25519,rsa-sha2-512,rsa-sha2-256,ssh-rsa [preauth] debug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com [preauth] debug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com [preauth] debug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1 [preauth] debug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1 [preauth] debug2: compression ctos: none,zlib@openssh.com,zlib [preauth] debug2: compression stoc: none,zlib@openssh.com,zlib [preauth] debug2: languages ctos: [preauth] debug2: languages stoc: [preauth] debug2: first_kex_follows 0 [preauth] debug2: reserved 0 [preauth] debug1: kex: algorithm: curve25519-sha256 [preauth] debug1: kex: host key algorithm: ecdsa-sha2-nistp256 [preauth] debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: compression: none [preauth] debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: compression: none [preauth] debug1: expecting SSH2_MSG_KEX_ECDH_INIT [preauth] debug3: receive packet: type 30 [preauth] debug3: mm_sshkey_sign entering [preauth] debug3: mm_request_send entering: type 6 [preauth] debug3: mm_sshkey_sign: waiting for MONITOR_ANS_SIGN [preauth] debug3: mm_request_receive_expect entering: type 7 [preauth] debug3: mm_request_receive entering [preauth] debug3: mm_request_receive entering debug3: monitor_read: checking request 6 debug3: mm_answer_sign debug3: mm_answer_sign: KEX signature 0x5650e7b4dbe0(100) debug3: mm_request_send entering: type 7 debug2: monitor_read: 6 used once, disabling now debug3: send packet: type 31 [preauth] debug3: send packet: type 21 [preauth] debug2: set_newkeys: mode 1 [preauth] debug1: rekey out after 134217728 blocks [preauth] debug1: SSH2_MSG_NEWKEYS sent [preauth] debug1: Sending SSH2_MSG_EXT_INFO [preauth] debug3: send packet: type 7 [preauth] debug1: expecting SSH2_MSG_NEWKEYS [preauth] debug3: receive packet: type 21 [preauth] debug1: SSH2_MSG_NEWKEYS received [preauth] debug2: set_newkeys: mode 0 [preauth] debug1: rekey in after 134217728 blocks [preauth] debug1: KEX done [preauth] debug3: receive packet: type 5 [preauth] debug3: send packet: type 6 [preauth] debug3: receive packet: type 50 [preauth] debug1: userauth-request for user domain\\testuser service ssh-connection method none [preauth] debug1: attempt 0 failures 0 [preauth] debug3: mm_getpwnamallow entering [preauth] debug3: mm_request_send entering: type 8 [preauth] debug3: mm_getpwnamallow: waiting for MONITOR_ANS_PWNAM [preauth] debug3: mm_request_receive_expect entering: type 9 [preauth] debug3: mm_request_receive entering [preauth] debug3: mm_request_receive entering debug3: monitor_read: checking request 8 debug3: mm_answer_pwnamallow debug2: parse_server_config_depth: config reprocess config len 626 debug2: parse_server_config_depth: config /etc/ssh/sshd_config.d/DOMAIN_Auth_Key.conf len 106 debug3: checking match for 'User svcQremote' user domain\testuser host 10.1.35.100 addr 10.1.35.100 laddr 10.1.27.48 lport 2222 debug3: match not found debug3: mm_answer_pwnamallow: sending MONITOR_ANS_PWNAM: 1 debug3: mm_request_send entering: type 9 debug2: monitor_read: 8 used once, disabling now debug2: input_userauth_request: setting up authctxt for domain\\testuser [preauth] debug3: mm_start_pam entering [preauth] debug3: mm_request_send entering: type 100 [preauth] debug3: mm_inform_authserv entering [preauth] debug3: mm_request_send entering: type 4 [preauth] debug3: mm_auth2_read_banner entering [preauth] debug3: mm_request_send entering: type 10 [preauth] debug3: mm_request_receive_expect entering: type 11 [preauth] debug3: mm_request_receive entering debug3: monitor_read: checking request 100 debug1: PAM: initializing for "domain\testuser" debug1: PAM: setting PAM_RHOST to "10.1.35.100" debug1: PAM: setting PAM_TTY to "ssh" debug2: monitor_read: 100 used once, disabling now debug3: mm_request_receive entering [preauth] debug3: mm_request_receive entering debug3: monitor_read: checking request 4 debug3: mm_answer_authserv: service=ssh-connection, style=, role= debug2: monitor_read: 4 used once, disabling now debug3: mm_request_receive entering debug3: monitor_read: checking request 10 debug3: mm_request_send entering: type 11 debug2: monitor_read: 10 used once, disabling now debug3: send packet: type 53 [preauth] debug1: userauth_send_banner: sent [preauth] debug2: input_userauth_request: try method none [preauth] debug3: user_specific_delay: user specific delay 0.000ms [preauth] debug3: ensure_minimum_time_since: elapsed 27.545ms, delaying 23.314ms (requested 6.357ms) [preauth] debug3: userauth_finish: failure partial=0 next methods="publickey,gssapi-keyex,gssapi-with-mic,password" [preauth] debug3: send packet: type 51 [preauth] debug3: receive packet: type 50 [preauth] debug1: userauth-request for user domain\\testuser service ssh-connection method gssapi-with-mic [preauth] debug1: attempt 1 failures 0 [preauth] debug2: input_userauth_request: try method gssapi-with-mic [preauth] debug3: mm_request_send entering: type 42 [preauth] debug3: mm_request_receive_expect entering: type 43 [preauth] debug3: mm_request_receive entering [preauth] debug3: mm_request_receive entering debug3: monitor_read: checking request 42 debug3: mm_request_send entering: type 43 debug3: send packet: type 60 [preauth] debug3: user_specific_delay: user specific delay 0.000ms [preauth] debug3: ensure_minimum_time_since: elapsed 6.443ms, delaying 6.272ms (requested 6.357ms) [preauth] Postponed gssapi-with-mic for domain\\testuser from 10.1.35.100 port 20520 ssh2 [preauth] debug3: receive packet: type 61 [preauth] debug3: mm_request_send entering: type 44 [preauth] debug3: mm_request_receive_expect entering: type 45 [preauth] debug3: mm_request_receive entering [preauth] debug3: mm_request_receive entering debug3: monitor_read: checking request 44 debug1: Got no client credentials debug3: mm_request_send entering: type 45 debug3: send packet: type 61 [preauth] debug3: receive packet: type 66 [preauth] debug3: mm_request_send entering: type 48 [preauth] debug3: mm_request_receive_expect entering: type 49 [preauth] debug3: mm_request_receive entering [preauth] debug3: mm_request_receive entering debug3: monitor_read: checking request 48 debug3: mm_request_send entering: type 49 debug2: monitor_read: 48 used once, disabling now debug3: mm_request_send entering: type 46 [preauth] debug3: mm_request_receive_expect entering: type 47 [preauth] debug3: mm_request_receive entering [preauth] debug3: mm_request_receive entering debug3: monitor_read: checking request 46 debug3: mm_answer_gss_userok: sending result 0 debug3: mm_request_send entering: type 47 debug2: monitor_read: 46 used once, disabling now Failed gssapi-with-mic for domain\testuser from 10.1.35.100 port 20520 ssh2 debug3: mm_ssh_gssapi_userok: user not authenticated [preauth] debug3: userauth_finish: failure partial=0 next methods="publickey,gssapi-keyex,gssapi-with-mic,password" [preauth] debug3: send packet: type 51 [preauth] Connection reset by authenticating user domain\\testuser 10.1.35.100 port 20520 [preauth] debug1: do_cleanup [preauth] debug3: PAM: sshpam_thread_cleanup entering [preauth] debug1: monitor_read_log: child log fd closed debug3: mm_request_receive entering debug1: do_cleanup debug1: PAM: cleanup debug3: PAM: sshpam_thread_cleanup entering debug1: Killing privsep child 82176 debug1: audit_event: unhandled event 12

SSHD CONFIG

Include /etc/ssh/sshd_config.d/*.conf LogLevel INFO

LoginGraceTime 60 PermitRootLogin no MaxAuthTries 4 MaxSessions 10

HostbasedAuthentication no IgnoreRhosts yes

PermitEmptyPasswords no

ChallengeResponseAuthentication no

GSSAPIAuthentication yes GSSAPIKeyExchange yes

UsePAM yes

X11Forwarding no PrintMotd no PermitUserEnvironment no ClientAliveInterval 300 ClientAliveCountMax 3 MaxStartups 10:30:60

Banner /etc/issue.net

AcceptEnv LANG LC_*

Subsystem sftp /usr/lib/openssh/sftp-server

PasswordAuthentication yes

debug1: Authenticating to clientdns:2222 as 'domain\testuser'

I suspect Linux does not like the domain\ prefix that Windows prepends to your username. As a workaround, specify the username explicitly when SSHing from Windows to Linux.

Once authentication works: to make sec=krb5 work, you probably also need to enable ticket delegation.

Tried with just the username and still failed:

ClientLog:

OpenSSH_for_Windows_8.1p1, LibreSSL 2.9.2 debug3: Failed to open file:C:/Users/admin/.ssh/config error:2 debug3: Failed to open file:C:/ProgramData/ssh/ssh_config error:2 debug2: resolving "clientdns" port 2222 debug2: ssh_connect_direct debug1: Connecting to clientdns [10.1.27.48] port 2222. debug1: Connection established. debug3: Failed to open file:C:/Users/admin/.ssh/id_rsa error:2 debug3: Failed to open file:C:/Users/admin/.ssh/id_rsa.pub error:2 debug1: identity file C:\Users\admin/.ssh/id_rsa type -1 debug3: Failed to open file:C:/Users/admin/.ssh/id_rsa-cert error:2 debug3: Failed to open file:C:/Users/admin/.ssh/id_rsa-cert.pub error:2 debug1: identity file C:\Users\admin/.ssh/id_rsa-cert type -1 debug3: Failed to open file:C:/Users/admin/.ssh/id_dsa error:2 debug3: Failed to open file:C:/Users/admin/.ssh/id_dsa.pub error:2 debug1: identity file C:\Users\admin/.ssh/id_dsa type -1 debug3: Failed to open file:C:/Users/admin/.ssh/id_dsa-cert error:2 debug3: Failed to open file:C:/Users/admin/.ssh/id_dsa-cert.pub error:2 debug1: identity file C:\Users\admin/.ssh/id_dsa-cert type -1 debug3: Failed to open file:C:/Users/admin/.ssh/id_ecdsa error:2 debug3: Failed to open file:C:/Users/admin/.ssh/id_ecdsa.pub error:2 debug1: identity file C:\Users\admin/.ssh/id_ecdsa type -1 debug3: Failed to open file:C:/Users/admin/.ssh/id_ecdsa-cert error:2 debug3: Failed to open file:C:/Users/admin/.ssh/id_ecdsa-cert.pub error:2 debug1: identity file C:\Users\admin/.ssh/id_ecdsa-cert type -1 debug3: Failed to open file:C:/Users/admin/.ssh/id_ed25519 error:2 debug3: Failed to open file:C:/Users/admin/.ssh/id_ed25519.pub error:2 debug1: identity file C:\Users\admin/.ssh/id_ed25519 type -1 debug3: Failed to open file:C:/Users/admin/.ssh/id_ed25519-cert error:2 debug3: Failed to open file:C:/Users/admin/.ssh/id_ed25519-cert.pub error:2 debug1: identity file C:\Users\admin/.ssh/id_ed25519-cert type -1 debug3: Failed to open file:C:/Users/admin/.ssh/id_xmss error:2 debug3: Failed to open file:C:/Users/admin/.ssh/id_xmss.pub error:2 debug1: identity file C:\Users\admin/.ssh/id_xmss type -1 debug3: Failed to open file:C:/Users/admin/.ssh/id_xmss-cert error:2 debug3: Failed to open file:C:/Users/admin/.ssh/id_xmss-cert.pub error:2 debug1: identity file C:\Users\admin/.ssh/id_xmss-cert type -1 debug1: Local version string SSH-2.0-OpenSSH_for_Windows_8.1 debug1: Remote protocol version 2.0, remote software version OpenSSH_8.2p1 Ubuntu-4ubuntu0.3 debug1: match: OpenSSH_8.2p1 Ubuntu-4ubuntu0.3 pat OpenSSH* compat 0x04000000 debug2: fd 3 setting O_NONBLOCK debug1: Authenticating to clientdns:2222 as 'testuser' debug3: put_host_port: [clientdns]:2222 debug3: hostkeys_foreach: reading file "C:\Users\admin/.ssh/known_hosts" debug3: Failed to open file:C:/Users/admin/.ssh/known_hosts2 error:2 debug3: Failed to open file:C:/ProgramData/ssh/ssh_known_hosts error:2 debug3: Failed to open file:C:/ProgramData/ssh/ssh_known_hosts2 error:2 debug3: send packet: type 20 debug1: SSH2_MSG_KEXINIT sent debug3: receive packet: type 20 debug1: SSH2_MSG_KEXINIT received debug2: local client KEXINIT proposal debug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,ext-info-c debug2: host key algorithms: ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ssh-ed25519-cert-v01@openssh.com,rsa-sha2-512-cert-v01@openssh.com,rsa-sha2-256-cert-v01@openssh.com,ssh-rsa-cert-v01@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-ed25519,rsa-sha2-512,rsa-sha2-256,ssh-rsa debug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com debug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com debug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1 debug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1 debug2: compression ctos: none,zlib@openssh.com,zlib debug2: compression stoc: none,zlib@openssh.com,zlib debug2: languages ctos: debug2: languages stoc: debug2: first_kex_follows 0 debug2: reserved 0 debug2: peer server KEXINIT proposal debug2: KEX algorithms: gss-gex-sha1-toWM5Slw5Ew8Mqkay+al2g==,gss-group14-sha1-toWM5Slw5Ew8Mqkay+al2g==,curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256 debug2: host key algorithms: rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519 debug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com debug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com debug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1 debug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1 debug2: compression ctos: none,zlib@openssh.com debug2: compression stoc: none,zlib@openssh.com debug2: languages ctos: debug2: languages stoc: debug2: first_kex_follows 0 debug2: reserved 0 debug1: kex: algorithm: curve25519-sha256 debug1: kex: host key algorithm: ecdsa-sha2-nistp256 debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: compression: none debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: compression: none debug3: send packet: type 30 debug1: expecting SSH2_MSG_KEX_ECDH_REPLY debug3: receive packet: type 31 debug1: Server host key: ecdsa-sha2-nistp256 SHA256:Tk6LVFbMXjnIJK1sP+9CWHcIzsOuDeeSDoKwPopmAmw debug3: put_host_port: [10.1.27.48]:2222 debug3: put_host_port: [clientdns]:2222 debug3: hostkeys_foreach: reading file "C:\Users\admin/.ssh/known_hosts" debug3: Failed to open file:C:/Users/admin/.ssh/known_hosts2 error:2 debug3: Failed to open file:C:/ProgramData/ssh/ssh_known_hosts error:2 debug3: Failed to open file:C:/ProgramData/ssh/ssh_known_hosts2 error:2 debug3: hostkeys_foreach: reading file "C:\Users\admin/.ssh/known_hosts" debug3: Failed to open file:C:/Users/admin/.ssh/known_hosts2 error:2 debug3: Failed to open file:C:/ProgramData/ssh/ssh_known_hosts error:2 debug3: Failed to open file:C:/ProgramData/ssh/ssh_known_hosts2 error:2 debug1: checking without port identifier debug3: hostkeys_foreach: reading file "C:\Users\admin/.ssh/known_hosts" debug3: record_hostkey: found key type ECDSA in file C:\Users\admin/.ssh/known_hosts:1 debug3: load_hostkeys: loaded 1 keys from clientdns debug3: Failed to open file:C:/Users/admin/.ssh/known_hosts2 error:2 debug3: Failed to open file:C:/ProgramData/ssh/ssh_known_hosts error:2 debug3: Failed to open file:C:/ProgramData/ssh/ssh_known_hosts2 error:2 debug3: hostkeys_foreach: reading file "C:\Users\admin/.ssh/known_hosts" debug3: record_hostkey: found key type ECDSA in file C:\Users\admin/.ssh/known_hosts:1 debug3: load_hostkeys: loaded 1 keys from 10.1.27.48 debug3: Failed to open file:C:/Users/admin/.ssh/known_hosts2 error:2 debug3: Failed to open file:C:/ProgramData/ssh/ssh_known_hosts error:2 debug3: Failed to open file:C:/ProgramData/ssh/ssh_known_hosts2 error:2 debug1: Host 'clientdns' is known and matches the ECDSA host key. debug1: Found key in C:\Users\admin/.ssh/known_hosts:1 debug1: found matching key w/out port debug3: send packet: type 21 debug2: set_newkeys: mode 1 debug1: rekey out after 134217728 blocks debug1: SSH2_MSG_NEWKEYS sent debug1: expecting SSH2_MSG_NEWKEYS debug3: receive packet: type 21 debug1: SSH2_MSG_NEWKEYS received debug2: set_newkeys: mode 0 debug1: rekey in after 134217728 blocks debug3: unable to connect to pipe \\.\pipe\openssh-ssh-agent, error: 2 debug1: pubkey_prepare: ssh_get_authentication_socket: No such file or directory debug1: Will attempt key: C:\Users\admin/.ssh/id_rsa debug1: Will attempt key: C:\Users\admin/.ssh/id_dsa debug1: Will attempt key: C:\Users\admin/.ssh/id_ecdsa debug1: Will attempt key: C:\Users\admin/.ssh/id_ed25519 debug1: Will attempt key: C:\Users\admin/.ssh/id_xmss debug2: pubkey_prepare: done debug3: send packet: type 5 debug3: receive packet: type 7 debug1: SSH2_MSG_EXT_INFO received debug1: kex_input_ext_info: server-sig-algs=ssh-ed25519,sk-ssh-ed25519@openssh.com,ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ecdsa-sha2-nistp256@openssh.com debug3: receive packet: type 6 debug2: service_accept: ssh-userauth debug1: SSH2_MSG_SERVICE_ACCEPT received debug3: send packet: type 50 debug3: receive packet: type 53 debug3: input_userauth_banner Authorized uses only. All activities performed on this system will be monitored. debug3: receive packet: type 51 debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password debug3: start over, passed a different list publickey,gssapi-keyex,gssapi-with-mic,password debug3: preferred gssapi-with-mic,publickey,keyboard-interactive,password debug3: authmethod_lookup gssapi-with-mic debug3: remaining preferred: publickey,keyboard-interactive,password debug3: authmethod_is_enabled gssapi-with-mic debug1: Next authentication method: gssapi-with-mic debug3: send packet: type 50 debug2: we sent a gssapi-with-mic packet, wait for reply debug3: receive packet: type 60 debug1: Delegating credentials debug1: sspi delegation was requested but not fulfilled debug3: send packet: type 61 debug3: receive packet: type 61 debug1: Delegating credentials debug1: sspi delegation was requested but not fulfilled debug3: send packet: type 66 debug3: receive packet: type 51 debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password debug2: we did not send a packet, disable method debug3: authmethod_lookup publickey debug3: remaining preferred: keyboard-interactive,password debug3: authmethod_is_enabled publickey debug1: Next authentication method: publickey debug1: Trying private key: C:\Users\admin/.ssh/id_rsa debug3: no such identity: C:\Users\admin/.ssh/id_rsa: No such file or directory debug1: Trying private key: C:\Users\admin/.ssh/id_dsa debug3: no such identity: C:\Users\admin/.ssh/id_dsa: No such file or directory debug1: Trying private key: C:\Users\admin/.ssh/id_ecdsa debug3: no such identity: C:\Users\admin/.ssh/id_ecdsa: No such file or directory debug1: Trying private key: C:\Users\admin/.ssh/id_ed25519 debug3: no such identity: C:\Users\admin/.ssh/id_ed25519: No such file or directory debug1: Trying private key: C:\Users\admin/.ssh/id_xmss debug3: no such identity: C:\Users\admin/.ssh/id_xmss: No such file or directory debug2: we did not send a packet, disable method debug3: authmethod_lookup password debug3: remaining preferred: ,password debug3: authmethod_is_enabled password debug1: Next authentication method: password debug3: failed to open file:C:/dev/tty error:3 debug1: read_passphrase: can't open /dev/tty: No such file or directory

ServerLog:

debug2: load_server_config: filename /etc/ssh/sshd_config debug2: load_server_config: done config len = 626 debug2: parse_server_config_depth: config /etc/ssh/sshd_config len 626 debug2: /etc/ssh/sshd_config line 13: new include /etc/ssh/sshd_config.d/.conf debug2: /etc/ssh/sshd_config line 13: including /etc/ssh/sshd_config.d/DOMAIN_Auth_Key.conf debug2: load_server_config: filename /etc/ssh/sshd_config.d/DOMAIN_Auth_Key.conf debug2: load_server_config: done config len = 106 debug2: parse_server_config_depth: config /etc/ssh/sshd_config.d/DOMAIN_Auth_Key.conf len 106 debug3: /etc/ssh/sshd_config.d/DOMAIN_Auth_Key.conf:1 setting AuthorizedKeysFile /etc/ssh/keys/%u/authorized_keys debug3: checking syntax for 'Match User svcQremote' debug3: /etc/ssh/sshd_config:29 setting LogLevel INFO debug3: /etc/ssh/sshd_config:34 setting LoginGraceTime 60 debug3: /etc/ssh/sshd_config:36 setting PermitRootLogin no debug3: /etc/ssh/sshd_config:39 setting MaxAuthTries 4 debug3: /etc/ssh/sshd_config:40 setting MaxSessions 10 debug3: /etc/ssh/sshd_config:54 setting HostbasedAuthentication no debug3: /etc/ssh/sshd_config:59 setting IgnoreRhosts yes debug3: /etc/ssh/sshd_config:63 setting PermitEmptyPasswords no debug3: /etc/ssh/sshd_config:67 setting ChallengeResponseAuthentication no debug3: /etc/ssh/sshd_config:76 setting GSSAPIAuthentication yes debug3: /etc/ssh/sshd_config:79 setting GSSAPIKeyExchange yes debug3: /etc/ssh/sshd_config:90 setting UsePAM yes debug3: /etc/ssh/sshd_config:96 setting X11Forwarding no debug3: /etc/ssh/sshd_config:100 setting PrintMotd no debug3: /etc/ssh/sshd_config:103 setting PermitUserEnvironment no debug3: /etc/ssh/sshd_config:106 setting ClientAliveInterval 300 debug3: /etc/ssh/sshd_config:107 setting ClientAliveCountMax 3 debug3: /etc/ssh/sshd_config:111 setting MaxStartups 10:30:60 debug3: /etc/ssh/sshd_config:118 setting Banner /etc/issue.net debug3: /etc/ssh/sshdconfig:121 setting AcceptEnv LANG LC debug3: /etc/ssh/sshd_config:124 setting Subsystem sftp /usr/lib/openssh/sftp-server debug3: /etc/ssh/sshd_config:132 setting PasswordAuthentication yes debug1: sshd version OpenSSH_8.2, OpenSSL 1.1.1f 31 Mar 2020 debug1: private host key #0: ssh-rsa SHA256:X4Nfc26X9hn15z+gWtrvqn0G54o4etTEl9gKcW/f6hI debug1: private host key #1: ecdsa-sha2-nistp256 SHA256:Tk6LVFbMXjnIJK1sP+9CWHcIzsOuDeeSDoKwPopmAmw debug1: private host key #2: ssh-ed25519 SHA256:6bqsDeIw6RSNeZ0JIU3QHZpLzDqz8lUIvwCzeiLXJTU debug1: rexec_argv[0]='/usr/sbin/sshd' debug1: rexec_argv[1]='-p' debug1: rexec_argv[2]='2222' debug1: rexec_argv[3]='-ddd' debug3: oom_adjust_setup debug1: Set /proc/self/oom_score_adj from 0 to -1000 debug2: fd 3 setting O_NONBLOCK debug1: Bind to port 2222 on 0.0.0.0. Server listening on 0.0.0.0 port 2222. debug2: fd 4 setting O_NONBLOCK debug3: sock_set_v6only: set socket 4 IPV6_V6ONLY debug1: Bind to port 2222 on ::. Server listening on :: port 2222. debug3: fd 5 is not O_NONBLOCK debug1: Server will not fork when running in debugging mode. debug3: send_rexec_state: entering fd = 8 config len 626 debug3: ssh_msg_send: type 0 debug3: send_rexec_state: done debug1: rexec start in 5 out 5 newsock 5 pipe -1 sock 8 debug3: recv_rexec_state: entering fd = 5 debug3: ssh_msg_recv entering debug3: recv_rexec_state: done debug2: parse_server_config_depth: config rexec len 626 debug2: parse_server_config_depth: config /etc/ssh/sshd_config.d/DOMAIN_Auth_Key.conf len 106 debug3: /etc/ssh/sshd_config.d/DOMAIN_Auth_Key.conf:1 setting AuthorizedKeysFile /etc/ssh/keys/%u/authorizedkeys debug3: checking syntax for 'Match User svcQremote' debug3: rexec:29 setting LogLevel INFO debug3: rexec:34 setting LoginGraceTime 60 debug3: rexec:36 setting PermitRootLogin no debug3: rexec:39 setting MaxAuthTries 4 debug3: rexec:40 setting MaxSessions 10 debug3: rexec:54 setting HostbasedAuthentication no debug3: rexec:59 setting IgnoreRhosts yes debug3: rexec:63 setting PermitEmptyPasswords no debug3: rexec:67 setting ChallengeResponseAuthentication no debug3: rexec:76 setting GSSAPIAuthentication yes debug3: rexec:79 setting GSSAPIKeyExchange yes debug3: rexec:90 setting UsePAM yes debug3: rexec:96 setting X11Forwarding no debug3: rexec:100 setting PrintMotd no debug3: rexec:103 setting PermitUserEnvironment no debug3: rexec:106 setting ClientAliveInterval 300 debug3: rexec:107 setting ClientAliveCountMax 3 debug3: rexec:111 setting MaxStartups 10:30:60 debug3: rexec:118 setting Banner /etc/issue.net debug3: rexec:121 setting AcceptEnv LANG LC debug3: rexec:124 setting Subsystem sftp /usr/lib/openssh/sftp-server debug3: rexec:132 setting PasswordAuthentication yes debug1: sshd version OpenSSH_8.2, OpenSSL 1.1.1f 31 Mar 2020 debug1: private host key #0: ssh-rsa SHA256:X4Nfc26X9hn15z+gWtrvqn0G54o4etTEl9gKcW/f6hI debug1: private host key #1: ecdsa-sha2-nistp256 SHA256:Tk6LVFbMXjnIJK1sP+9CWHcIzsOuDeeSDoKwPopmAmw debug1: private host key #2: ssh-ed25519 SHA256:6bqsDeIw6RSNeZ0JIU3QHZpLzDqz8lUIvwCzeiLXJTU debug1: inetd sockets after dupping: 3, 3 Connection from 10.1.35.100 port 20485 on 10.1.27.48 port 2222 rdomain "" debug1: Local version string SSH-2.0-OpenSSH_8.2p1 Ubuntu-4ubuntu0.3 debug1: Remote protocol version 2.0, remote software version OpenSSH_for_Windows_8.1 debug1: match: OpenSSH_for_Windows_8.1 pat OpenSSH compat 0x04000000 debug2: fd 3 setting O_NONBLOCK debug3: ssh_sandbox_init: preparing seccomp filter sandbox debug2: Network child is on pid 398042 debug3: preauth child monitor started debug3: privsep user:group 112:65534 [preauth] debug1: permanently_set_uid: 112/65534 [preauth] debug3: ssh_sandbox_child: setting PR_SET_NO_NEW_PRIVS [preauth] debug3: ssh_sandbox_child: attaching seccomp filter program [preauth] debug1: list_hostkey_types: rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519 [preauth] debug3: mm_request_send entering: type 42 [preauth] debug3: mm_request_receive_expect entering: type 43 [preauth] debug3: mm_request_receive entering [preauth] debug3: mm_request_receive entering debug3: monitor_read: checking request 42 debug3: mm_request_send entering: type 43 debug3: send packet: type 20 [preauth] debug1: SSH2_MSG_KEXINIT sent [preauth] debug3: receive packet: type 20 [preauth] debug1: SSH2_MSG_KEXINIT received [preauth] debug2: local server KEXINIT proposal [preauth] debug2: KEX algorithms: gss-gex-sha1-toWM5Slw5Ew8Mqkay+al2g==,gss-group14-sha1-toWM5Slw5Ew8Mqkay+al2g==,curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256 [preauth] debug2: host key algorithms: rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519 [preauth] debug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com [preauth] debug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com [preauth] debug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1 [preauth] debug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1 [preauth] debug2: compression ctos: none,zlib@openssh.com [preauth] debug2: compression stoc: none,zlib@openssh.com [preauth] debug2: languages ctos: [preauth] debug2: languages stoc: [preauth] debug2: first_kex_follows 0 [preauth] debug2: reserved 0 [preauth] debug2: peer client KEXINIT proposal [preauth] debug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,ext-info-c [preauth] debug2: host key algorithms: ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ssh-ed25519-cert-v01@openssh.com,rsa-sha2-512-cert-v01@openssh.com,rsa-sha2-256-cert-v01@openssh.com,ssh-rsa-cert-v01@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-ed25519,rsa-sha2-512,rsa-sha2-256,ssh-rsa [preauth] debug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com [preauth] debug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com [preauth] debug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1 [preauth] debug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1 [preauth] debug2: compression ctos: none,zlib@openssh.com,zlib [preauth] debug2: compression stoc: none,zlib@openssh.com,zlib [preauth] debug2: languages ctos: [preauth] debug2: languages stoc: [preauth] debug2: first_kex_follows 0 [preauth] debug2: reserved 0 [preauth] debug1: kex: algorithm: curve25519-sha256 [preauth] debug1: kex: host key algorithm: ecdsa-sha2-nistp256 [preauth] debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: compression: none [preauth] debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: compression: none [preauth] debug1: expecting SSH2_MSG_KEX_ECDH_INIT [preauth] debug3: receive packet: type 30 [preauth] debug3: mm_sshkey_sign entering [preauth] debug3: mm_request_send entering: type 6 [preauth] debug3: mm_sshkey_sign: waiting for MONITOR_ANS_SIGN [preauth] debug3: mm_request_receive_expect entering: type 7 [preauth] debug3: mm_request_receive entering [preauth] debug3: mm_request_receive entering debug3: monitor_read: checking request 6 debug3: mm_answer_sign debug3: mm_answer_sign: KEX signature 0x55aa6c099be0(99) debug3: mm_request_send entering: type 7 debug2: monitor_read: 6 used once, disabling now debug3: send packet: type 31 [preauth] debug3: send packet: type 21 [preauth] debug2: set_newkeys: mode 1 [preauth] debug1: rekey out after 134217728 blocks [preauth] debug1: SSH2_MSG_NEWKEYS sent [preauth] debug1: Sending SSH2_MSG_EXT_INFO [preauth] debug3: send packet: type 7 [preauth] debug1: expecting SSH2_MSG_NEWKEYS [preauth] debug3: receive packet: type 21 [preauth] debug1: SSH2_MSG_NEWKEYS received [preauth] debug2: set_newkeys: mode 0 [preauth] debug1: rekey in after 134217728 blocks [preauth] debug1: KEX done [preauth] debug3: receive packet: type 5 [preauth] debug3: send packet: type 6 [preauth] debug3: receive packet: type 50 [preauth] debug1: userauth-request for user testuser service ssh-connection method none [preauth] debug1: attempt 0 failures 0 [preauth] debug3: mm_getpwnamallow entering [preauth] debug3: mm_request_send entering: type 8 [preauth] debug3: mm_getpwnamallow: waiting for MONITOR_ANS_PWNAM [preauth] debug3: mm_request_receive_expect entering: type 9 [preauth] debug3: mm_request_receive entering [preauth] debug3: mm_request_receive entering debug3: monitor_read: checking request 8 debug3: mm_answer_pwnamallow debug2: parse_server_config_depth: config reprocess config len 626 debug2: parse_server_config_depth: config /etc/ssh/sshd_config.d/DOMAIN_Auth_Key.conf len 106 debug3: checking match for 'User svcQremote' user testuser host 10.1.35.100 addr 10.1.35.100 laddr 10.1.27.48 lport 2222 debug3: match not found debug3: mm_answer_pwnamallow: sending MONITOR_ANS_PWNAM: 1 debug3: mm_request_send entering: type 9 debug2: monitor_read: 8 used once, disabling now debug2: input_userauth_request: setting up authctxt for testuser [preauth] debug3: mm_start_pam entering [preauth] debug3: mm_request_send entering: type 100 [preauth] debug3: mm_inform_authserv entering [preauth] debug3: mm_request_send entering: type 4 [preauth] debug3: mm_auth2_read_banner entering [preauth] debug3: mm_request_send entering: type 10 [preauth] debug3: mm_request_receive_expect entering: type 11 [preauth] debug3: mm_request_receive entering [preauth] debug3: mm_request_receive entering debug3: monitor_read: checking request 100 debug1: PAM: initializing for "testuser" debug1: PAM: setting PAM_RHOST to "10.1.35.100" debug1: PAM: setting PAM_TTY to "ssh" debug2: monitor_read: 100 used once, disabling now debug3: mm_request_receive entering debug3: monitor_read: checking request 4 debug3: mm_answer_authserv: service=ssh-connection, style=, role= debug2: monitor_read: 4 used once, disabling now debug3: mm_request_receive entering debug3: monitor_read: checking request 10 debug3: mm_request_send entering: type 11 debug2: monitor_read: 10 used once, disabling now debug3: send packet: type 53 [preauth] debug1: userauth_send_banner: sent [preauth] debug2: input_userauth_request: try method none [preauth] debug3: user_specific_delay: user specific delay 0.000ms [preauth] debug3: ensure_minimum_time_since: elapsed 31.534ms, delaying 10.395ms (requested 5.241ms) [preauth] debug3: userauth_finish: failure partial=0 next methods="publickey,gssapi-keyex,gssapi-with-mic,password" [preauth] debug3: send packet: type 51 [preauth] debug3: receive packet: type 50 [preauth] debug1: userauth-request for user testuser service ssh-connection method gssapi-with-mic [preauth] debug1: attempt 1 failures 0 [preauth] debug2: input_userauth_request: try method gssapi-with-mic [preauth] debug3: mm_request_send entering: type 42 [preauth] debug3: mm_request_receive_expect entering: type 43 [preauth] debug3: mm_request_receive entering [preauth] debug3: mm_request_receive entering debug3: monitor_read: checking request 42 debug3: mm_request_send entering: type 43 debug3: send packet: type 60 [preauth] debug3: user_specific_delay: user specific delay 0.000ms [preauth] debug3: ensure_minimum_time_since: elapsed 4.674ms, delaying 0.567ms (requested 5.241ms) [preauth] Postponed gssapi-with-mic for testuser from 10.1.35.100 port 20485 ssh2 [preauth] debug3: receive packet: type 61 [preauth] debug3: mm_request_send entering: type 44 [preauth] debug3: mm_request_receive_expect entering: type 45 [preauth] debug3: mm_request_receive entering [preauth] debug3: mm_request_receive entering debug3: monitor_read: checking request 44 debug1: Got no client credentials debug3: mm_request_send entering: type 45 debug3: send packet: type 61 [preauth] debug3: receive packet: type 66 [preauth] debug3: mm_request_send entering: type 48 [preauth] debug3: mm_request_receive_expect entering: type 49 [preauth] debug3: mm_request_receive entering [preauth] debug3: mm_request_receive entering debug3: monitor_read: checking request 48 debug3: mm_request_send entering: type 49 debug2: monitor_read: 48 used once, disabling now debug3: mm_request_send entering: type 46 [preauth] debug3: mm_request_receive entering debug3: monitor_read: checking request 46 debug3: mm_answer_gss_userok: sending result 0 debug3: mm_request_send entering: type 47 debug2: monitor_read: 46 used once, disabling now Failed gssapi-with-mic for testuser from 10.1.35.100 port 20485 ssh2 debug3: mm_request_receive_expect entering: type 47 [preauth] debug3: mm_request_receive entering [preauth] debug3: mm_ssh_gssapi_userok: user not authenticated [preauth] debug3: userauth_finish: failure partial=0 next methods="publickey,gssapi-keyex,gssapi-with-mic,password" [preauth] debug3: send packet: type 51 [preauth] Connection reset by authenticating user testuser 10.1.35.100 port 20485 [preauth] debug1: do_cleanup [preauth] debug3: PAM: sshpam_thread_cleanup entering [preauth] debug1: monitor_read_log: child log fd closed debug3: mm_request_receive entering debug1: do_cleanup debug1: PAM: cleanup debug3: PAM: sshpam_thread_cleanup entering debug1: Killing privsep child 398042 debug1: audit_event: unhandled event 12

Seems to fail at the preauthentication. Is it a Microsoft SSPI thing? If so, where can I get support from for this?

@pengsiya

Can you try again with just GSSAPIAuthentication = Yes? Also does the hostname you're connecting to match the host name in Active Directory (or, if not, does an SPN exist on the AD object for any IP or hostname alias)? Do you see the Kerberos token cached when you do a 'klist' on the client system? Wireshark can be good for snooping Kerberos request issues.

As an additional data point, you could try PuTTY with GSSAPI enabled just to eliminate any OpenSSH client GSSAPI nuances.

@NoMoreFood

I tried with just GSSAPIAuthentication = Yes and got the same error.

The hostname should match with the host name in AD. I am able to use password authentication against AD. I tried with FQDN as well and no luck.

klist shows the following (omitted most contents)

2> Client: testuser @ DOMAIN

    Server: host/svrdns.DOMAIN @ DOMAIN
    KerbTicket Encryption Type: AES-256-CTS-HMAC-SHA1-96
    Ticket Flags 0x40a10000 -> forwardable renewable pre_authent name_canonicalize
    Start Time: 1/4/2022 23:17:09 (local)
    End Time:   1/5/2022 9:11:07 (local)
    Renew Time: 1/11/2022 23:11:07 (local)
    Session Key Type: AES-256-CTS-HMAC-SHA1-96
    Cache Flags: 0
    Kdc Called: dc.DOMAIN

With Putty, I got a similar error: Event Log: GSSAPI authentication initialised Event Log: GSSAPI authentication loop finished OK Outgoing packet #0x7, type 66 / 0x42 (SSH2_MSG_USERAUTH_GSSAPI_MIC) 00000000 00 00 00 1c 04 04 04 ff ff ff ff ff 00 00 00 00 ................ 00000010 6a a0 43 a7 d8 0b 42 9f fc 4a 81 9f 96 99 1a 61 j.C...B..J.....a Incoming packet #0x8, type 51 / 0x33 (SSH2_MSG_USERAUTH_FAILURE)

I will give it a try with wireshark.

Thanks!

Just for reference, when I tried to connect a linux client to the linux server via GSSAPI, it also failed, even though both machines have active login session open and live kerberos tickets. So as I mentioned before this may not be an Win32-OpenSSH specific issue, but a general GSSAPI issue (or SSPI issue).

I really just want to know where I can get support for this issue. Any reply would be appreciated!!

here is a trimmed version of the ssh logs for Linux -> Linux GSSAPI auth attempt:

Client:

debug1: SSH2_MSG_SERVICE_ACCEPT received debug3: send packet: type 50 debug3: receive packet: type 53 debug3: input_userauth_banner Authorized uses only. All activities performed on this system will be monitored. debug3: receive packet: type 51 debug1: Authentications that can continue: gssapi-keyex,gssapi-with-mic debug3: start over, passed a different list gssapi-keyex,gssapi-with-mic debug3: preferred gssapi-with-mic,publickey,keyboard-interactive,password debug3: authmethod_lookup gssapi-with-mic debug3: remaining preferred: publickey,keyboard-interactive,password debug3: authmethod_is_enabled gssapi-with-mic debug1: Next authentication method: gssapi-with-mic debug3: send packet: type 50 debug2: we sent a gssapi-with-mic packet, wait for reply debug3: receive packet: type 60 debug1: Delegating credentials debug3: send packet: type 61 debug3: receive packet: type 61 debug1: Delegating credentials debug3: send packet: type 66 debug3: receive packet: type 51 debug1: Authentications that can continue: gssapi-keyex,gssapi-with-mic debug3: send packet: type 50 debug2: we sent a gssapi-with-mic packet, wait for reply debug3: receive packet: type 51 debug1: Authentications that can continue: gssapi-keyex,gssapi-with-mic debug2: we did not send a packet, disable method debug1: No more authentication methods to try. testuser@svrdns: Permission denied (gssapi-keyex,gssapi-with-mic).

Server:

debug3: send packet: type 53 [preauth] debug1: userauth_send_banner: sent [preauth] debug2: input_userauth_request: try method none [preauth] debug3: user_specific_delay: user specific delay 0.000ms [preauth] debug3: ensure_minimum_time_since: elapsed 31.513ms, delaying 20.344ms (requested 6.482ms) [preauth] debug3: userauth_finish: failure partial=0 next methods="gssapi-keyex,gssapi-with-mic" [preauth] debug3: send packet: type 51 [preauth] debug3: receive packet: type 50 [preauth] debug1: userauth-request for user testuser service ssh-connection method gssapi-with-mic [preauth] debug1: attempt 1 failures 0 [preauth] debug2: input_userauth_request: try method gssapi-with-mic [preauth] debug3: mm_request_send entering: type 42 [preauth] debug3: mm_request_receive_expect entering: type 43 [preauth] debug3: mm_request_receive entering [preauth] debug3: mm_request_receive entering debug3: monitor_read: checking request 42 debug3: mm_request_send entering: type 43 debug3: send packet: type 60 [preauth] debug3: user_specific_delay: user specific delay 0.000ms [preauth] debug3: ensure_minimum_time_since: elapsed 5.676ms, delaying 0.807ms (requested 6.482ms) [preauth] Postponed gssapi-with-mic for testuser from 10.1.27.104 port 63924 ssh2 [preauth] debug3: receive packet: type 61 [preauth] debug3: mm_request_send entering: type 44 [preauth] debug3: mm_request_receive_expect entering: type 45 [preauth] debug3: mm_request_receive entering [preauth] debug3: mm_request_receive entering debug3: monitor_read: checking request 44 debug1: Got no client credentials debug3: mm_request_send entering: type 45 debug3: send packet: type 61 [preauth] debug3: receive packet: type 66 [preauth] debug3: mm_request_send entering: type 48 [preauth] debug3: mm_request_receive_expect entering: type 49 [preauth] debug3: mm_request_receive entering [preauth] debug3: mm_request_receive entering debug3: monitor_read: checking request 48 debug3: mm_request_send entering: type 49 debug2: monitor_read: 48 used once, disabling now debug3: mm_request_send entering: type 46 [preauth] debug3: mm_request_receive_expect entering: type 47 [preauth] debug3: mm_request_receive entering [preauth] debug3: mm_request_receive entering debug3: monitor_read: checking request 46 debug3: mm_answer_gss_userok: sending result 0 debug3: mm_request_send entering: type 47 debug2: monitor_read: 46 used once, disabling now Failed gssapi-with-mic for testuser from 10.1.27.104 port 63924 ssh2 debug3: mm_ssh_gssapi_userok: user not authenticated [preauth] debug3: userauth_finish: failure partial=0 next methods="gssapi-keyex,gssapi-with-mic" [preauth] debug3: send packet: type 51 [preauth] debug3: receive packet: type 50 [preauth] debug1: userauth-request for user testuser service ssh-connection method gssapi-with-mic [preauth] debug1: attempt 2 failures 1 [preauth] debug2: input_userauth_request: try method gssapi-with-mic [preauth] debug3: user_specific_delay: user specific delay 0.000ms [preauth] debug3: ensure_minimum_time_since: elapsed 0.064ms, delaying 6.419ms (requested 6.482ms) [preauth] debug3: userauth_finish: failure partial=0 next methods="gssapi-keyex,gssapi-with-mic" [preauth] debug3: send packet: type 51 [preauth] Connection closed by authenticating user testuser 10.1.27.104 port 63924 [preauth]

The only thing I found in my AD record for the linux machine is that the "Computer name" is full capital letters (no FQDN) but the "DNS name" is all lower cases (with FQDN).

Could this be an issue?

On the Linux server, what does sudo klist -kte show? It must list a host/hostname.fqdn@DOMAIN service principal name (SPN) for the server, stored in /etc/krb5.keytab. You must have exported from the Active Directory server using ktpass a keytab file for the server and stored it on the Linux server in /etc/krb5.keytab (only readable for root!), for /sbin/sshd to use for its end of the authentication.

On a Linux client, what does KRB5_TRACE=/dev/stdout ssh hostname.fqdn output?

What does klist show afterwards?

That same SPN (possibly without the @DOMAIN suffix) must also be found in an servicePrincipalName attribute in the Active Directory LDAP object representing the Linux server (can be a normal user entry, no need for it to actually be a computer entry), and the hostname.fqdn should also match what a rDNS lookup of the IP address of the SSH server resolves to. Because SSPI will search for the SPN (and Kerberos key) of the SSH server via its hostname in servicePrincipalName.

I should add that I have never used, and am not familiar with, sssd. At my department, we just manually create a user object in Active Directory for each Linux host, set the servicePrincipalName of that user object to host/hostname.fqdn, generate a key, export it into a keytab file with ktpass and store it on the Linux server in /etc/krb5.keytab. That's normally all that's needed to cause sshd on a Linux server to use be able to use Active Directory as a Kerberos5 KDC for GSSAPIAuthentication (assuming that the Linux server has already its own getent passwd and getent group user/group databases, and you don't need Active Directory for that as well).

I understand that there are other ways to “join” a Linux server to an Active Directory domain. They all need to somehow generate a Kerberos password for the server, and locally store the corresponding keytab file (= password-derived symmetric key). Some such techniques (e.g. e.g. Samba's net join) store that keytab elsewhere than /etc/krb5.keytab, and thus sshd might not find it (unless told about the new location via /etc/krb5.conf).

So make sure you understand where sssd keeps the SPN keytab that sshd is meant to use.

On the Linux server, what does sudo klist -kte show? It must list a host/hostname.fqdn@DOMAIN service principal name (SPN) for the server, stored in /etc/krb5.keytab.

Here is the output of klist -kte:

Keytab name: FILE:/etc/krb5.keytab
KVNO Timestamp           Principal
---- ------------------- ------------------------------------------------------
  12 12/17/2021 14:24:49 SVRDNS$@DOMAIN (DEPRECATED:arcfour-hmac)
  12 12/17/2021 14:24:49 SVRDNS$@DOMAIN (aes128-cts-hmac-sha1-96)
  12 12/17/2021 14:24:49 SVRDNS$@DOMAIN (aes256-cts-hmac-sha1-96)
  12 12/17/2021 14:24:49 host/SVRDNS@DOMAIN (DEPRECATED:arcfour-hmac)
  12 12/17/2021 14:24:49 host/SVRDNS@DOMAIN (aes128-cts-hmac-sha1-96)
  12 12/17/2021 14:24:49 host/SVRDNS@DOMAIN (aes256-cts-hmac-sha1-96)
  12 12/17/2021 14:24:49 host/svcdns.domain@DOMAIN (DEPRECATED:arcfour-hmac)
  12 12/17/2021 14:24:49 host/svcdns.domain@DOMAIN (aes128-cts-hmac-sha1-96)
  12 12/17/2021 14:24:49 host/svcdns.domain@DOMAIN (aes256-cts-hmac-sha1-96)
  12 12/17/2021 14:24:49 RestrictedKrbHost/SVRDNS@DOMAIN (DEPRECATED:arcfour-hmac)
  12 12/17/2021 14:24:49 RestrictedKrbHost/SVRDNS@DOMAIN (aes128-cts-hmac-sha1-96)
  12 12/17/2021 14:24:49 RestrictedKrbHost/SVRDNS@DOMAIN (aes256-cts-hmac-sha1-96)
  12 12/17/2021 14:24:49 RestrictedKrbHost/svcdns.domain@DOMAIN (DEPRECATED:arcfour-hmac)
  12 12/17/2021 14:24:49 RestrictedKrbHost/svcdns.domain@DOMAIN (aes128-cts-hmac-sha1-96)
  12 12/17/2021 14:24:49 RestrictedKrbHost/svcdns.domain@DOMAIN (aes256-cts-hmac-sha1-96)
  11 11/16/2021 22:33:25 SVRDNS$@DOMAIN (DEPRECATED:arcfour-hmac)
  11 11/16/2021 22:33:25 SVRDNS$@DOMAIN (aes128-cts-hmac-sha1-96)
  11 11/16/2021 22:33:25 SVRDNS$@DOMAIN (aes256-cts-hmac-sha1-96)
  11 11/16/2021 22:33:25 host/SVRDNS@DOMAIN (DEPRECATED:arcfour-hmac)
  11 11/16/2021 22:33:25 host/SVRDNS@DOMAIN (aes128-cts-hmac-sha1-96)
  11 11/16/2021 22:33:25 host/SVRDNS@DOMAIN (aes256-cts-hmac-sha1-96)
  11 11/16/2021 22:33:25 host/svcdns.domain@DOMAIN (DEPRECATED:arcfour-hmac)
  11 11/16/2021 22:33:25 host/svcdns.domain@DOMAIN (aes128-cts-hmac-sha1-96)
  11 11/16/2021 22:33:25 host/svcdns.domain@DOMAIN (aes256-cts-hmac-sha1-96)
  11 11/16/2021 22:33:25 RestrictedKrbHost/SVRDNS@DOMAIN (DEPRECATED:arcfour-hmac)
  11 11/16/2021 22:33:25 RestrictedKrbHost/SVRDNS@DOMAIN (aes128-cts-hmac-sha1-96)
  11 11/16/2021 22:33:25 RestrictedKrbHost/SVRDNS@DOMAIN (aes256-cts-hmac-sha1-96)
  11 11/16/2021 22:33:25 RestrictedKrbHost/svcdns.domain@DOMAIN (DEPRECATED:arcfour-hmac)
  11 11/16/2021 22:33:25 RestrictedKrbHost/svcdns.domain@DOMAIN (aes128-cts-hmac-sha1-96)
  11 11/16/2021 22:33:25 RestrictedKrbHost/svcdns.domain@DOMAIN (aes256-cts-hmac-sha1-96)

You must have exported from the Active Directory server using ktpass a keytab file for the server and stored it on the Linux server in /etc/krb5.keytab (only readable for root!), for /sbin/sshd to use for its end of the authentication.

The keytab was not exported from Active Directory. They were automatically generated by sssd I think when the server is joined to domain (via realm join/adcli)

On a Linux client, what does KRB5_TRACE=/dev/stdout ssh hostname.fqdn output?

[3324776] 1641422550.803248: ccselect module realm chose cache FILE:/tmp/krb5cc_892039766_jhfnGl with client principal testuser@DOMAIN for server principal host/svcdns.domain@DOMAIN [3324776] 1641422550.803249: Getting credentials testuser@DOMAIN -> host/svcdns.domain@ using ccache FILE:/tmp/krb5cc_892039766_jhfnGl [3324776] 1641422550.803250: Retrieving testuser@DOMAIN -> host/svcdns.domain@ from FILE:/tmp/krb5cc_892039766_jhfnGl with result: -1765328243/Matching credential not found (filename: /tmp/krb5cc_892039766_jhfnGl) [3324776] 1641422550.803251: Retrying testuser@DOMAIN -> host/svcdns.domain@DOMAIN with result: -1765328243/Matching credential not found (filename: /tmp/krb5cc_892039766_jhfnGl) [3324776] 1641422550.803252: Server has referral realm; starting with host/svcdns.domain@DOMAIN [3324776] 1641422550.803253: Retrieving testuser@DOMAIN -> krbtgt/DOMAIN@DOMAIN from FILE:/tmp/krb5cc_892039766_jhfnGl with result: 0/Success [3324776] 1641422550.803254: Starting with TGT for client realm: testuser@DOMAIN -> krbtgt/DOMAIN@DOMAIN [3324776] 1641422550.803255: Requesting tickets for host/svcdns.domain@DOMAIN, referrals on [3324776] 1641422550.803256: Generated subkey for TGS request: aes256-cts/4973 [3324776] 1641422550.803257: etypes requested in TGS request: aes256-cts, aes128-cts, aes256-sha2, aes128-sha2, des3-cbc-sha1, rc4-hmac, camellia128-cts, camellia256-cts [3324776] 1641422550.803259: Encoding request body and padata into FAST request [3324776] 1641422550.803260: Sending request (3021 bytes) to DOMAIN [3324776] 1641422550.803261: Initiating TCP connection to stream 10.1.0.51:88 [3324776] 1641422550.803262: Sending TCP request to stream 10.1.0.51:88 [3324776] 1641422550.803263: Received answer (2952 bytes) from stream 10.1.0.51:88 [3324776] 1641422550.803264: Terminating TCP connection to stream 10.1.0.51:88 [3324776] 1641422550.803265: Response was from master KDC [3324776] 1641422550.803266: Decoding FAST response [3324776] 1641422550.803267: FAST reply key: aes256-cts/577D [3324776] 1641422550.803268: TGS reply is for testuser@DOMAIN -> host/svcdns.domain@DOMAIN with session key aes256-cts/3ADD [3324776] 1641422550.803269: TGS request result: 0/Success [3324776] 1641422550.803270: Received creds for desired service host/svcdns.domain@DOMAIN [3324776] 1641422550.803271: Storing testuser@DOMAIN -> host/svcdns.domain@ in FILE:/tmp/krb5cc_892039766_jhfnGl [3324776] 1641422550.803272: Also storing testuser@DOMAIN -> host/svcdns.domain@DOMAIN based on ticket [3324776] 1641422550.803273: Removing testuser@DOMAIN -> host/svcdns.domain@DOMAIN from FILE:/tmp/krb5cc_892039766_jhfnGl [3324776] 1641422550.803275: Creating authenticator for testuser@DOMAIN -> host/svcdns.domain@, seqnum 872327293, subkey aes256-cts/D403, session key aes256-cts/3ADD [3324776] 1641422551.8767: ccselect module realm chose cache FILE:/tmp/krb5cc_892039766_jhfnGl with client principal testuser@DOMAIN for server principal host/svcdns.domain@DOMAIN [3324776] 1641422551.8768: Getting credentials testuser@DOMAIN -> host/svcdns.domain@ using ccache FILE:/tmp/krb5cc_892039766_jhfnGl [3324776] 1641422551.8769: Retrieving testuser@DOMAIN -> host/svcdns.domain@ from FILE:/tmp/krb5cc_892039766_jhfnGl with result: 0/Success [3324776] 1641422551.8771: Creating authenticator for testuser@DOMAIN -> host/svcdns.domain@, seqnum 279037212, subkey aes256-cts/8835, session key aes256-cts/3ADD [3324776] 1641422551.8776: Read AP-REP, time 1641422551.8772, subkey aes256-cts/616E, seqnum 130746151 [3324776] 1641422551.8781: ccselect module realm chose cache FILE:/tmp/krb5cc_892039766_jhfnGl with client principal testuser@DOMAIN for server principal host/svcdns.domain@DOMAIN [3324776] 1641422551.8782: Getting credentials testuser@DOMAIN -> host/svcdns.domain@ using ccache FILE:/tmp/krb5cc_892039766_jhfnGl [3324776] 1641422551.8783: Retrieving testuser@DOMAIN -> host/svcdns.domain@ from FILE:/tmp/krb5cc_892039766_jhfnGl with result: 0/Success [3324776] 1641422551.8785: Getting credentials testuser@DOMAIN -> host/svcdns.domain@ using ccache FILE:/tmp/krb5cc_892039766_jhfnGl [3324776] 1641422551.8786: Retrieving testuser@DOMAIN -> host/svcdns.domain@ from FILE:/tmp/krb5cc_892039766_jhfnGl with result: 0/Success [3324776] 1641422551.8788: Creating authenticator for testuser@DOMAIN -> host/svcdns.domain@, seqnum 597380254, subkey aes256-cts/C26B, session key aes256-cts/3ADD testuser@svcdns.domain: Permission denied (gssapi-keyex,gssapi-with-mic).

see highlighted line:

[3324776] 1641422550.803273: Removing testuser@DOMAIN -> host/svcdns.domain@DOMAIN from FILE:/tmp/krb5cc_892039766_jhfnGl

why is that credential removed?

What does klist show afterwards?

identical as before

That same SPN (possibly without the @DOMAIN suffix) must also be found in an servicePrincipalName attribute in the Active Directory LDAP object representing the Linux server (can be a normal user entry, no need for it to actually be a computer entry), and the hostname.fqdn should also match what a rDNS lookup of the IP address of the SSH server resolves to. Because SSPI will search for the SPN (and Kerberos key) of the SSH server via its hostname in servicePrincipalName.

The linux server is in AD as a computer entry. When I query for the SPN for the server:

setspn -l svcdns Registered ServicePrincipalNames for CN=svcdns,OU=Linux,OU=,OU=Servers,DC=,DC=***: RestrictedKrbHost/svrdns.domain RestrictedKrbHost/SVRDNS host/svrdns.domain host/SVRDNS

I should add that I have never used, and am not familiar with, sssd. At my department, we just manually create a user object in Active Directory for each Linux host, set the servicePrincipalName of that user object to host/hostname.fqdn, generate a key, export it into a keytab file with ktpass and store it on the Linux server in /etc/krb5.keytab. That's normally all that's needed to cause sshd on a Linux server to use be able to use Active Directory as a Kerberos5 KDC for GSSAPIAuthentication (assuming that the Linux server has already its own getent passwd and getent group user/group databases, and you don't need Active Directory for that as well).

I understand that there are other ways to “join” a Linux server to an Active Directory domain. They all need to somehow generate a Kerberos password for the server, and locally store the corresponding keytab file (= password-derived symmetric key). Some such techniques (e.g. e.g. Samba's net join) store that keytab elsewhere than /etc/krb5.keytab, and thus sshd might not find it (unless told about the new location via /etc/krb5.conf).

So make sure you understand where sssd keeps the SPN keytab that sshd is meant to use.

Thanks @mgkuhn for the followup, much appreciated!

I personally don't have a lot of experience using sssd and in fact our company was just trying to move onto a hybrid platform environment (in the past pure Windows).

One thing I can tell for sure is that on the linux machine (both client and server), I can use the kerberos ticket to do smbclient connection:

(base) testuser@cltdns:~$ smbclient -k //host.fqdn/share
Try "help" to get a list of possible commands.
smb: \>

So the keytab on the server should be somewhat working.

servicePrincipalName isn't something that we control and when I join the machine to AD I didn't have to specify it.

Also I found using the utility "setspn -l" that our machine account generally has SPN but none of the user account has SPN by default.

On any Linux machine (that can contact port 88 of the domain controller), can you get as a normal user with

$ kvno host/svrdns.domain
$ klist

a ticket for principal host/svrdns.domain?

If not, does

$ kvno host/svrdns.domain@DOMAIN
$ klist

work?

The kvno command command allows you to test getting a ticket, without actually using it. I think on Windows the klist get option, as in

C:\> klist get host/svrdns.domain
C:\> klist

might do the same.

see highlighted line:

[3324776] 1641422550.803273: Removing testuser@DOMAIN -> host/svcdns.domain@DOMAIN from FILE:/tmp/krb5cc_892039766_jhfnGl

why is that credential removed?

I don't understand it, but I can see it as well in a normally working session, so I think that's normal and not an indication of any problem.

I think the line

[3324776] 1641422551.8788: Creating authenticator for testuser@DOMAIN -> host/svcdns.domain@, seqnum 597380254, subkey aes256-cts/C26B, session key aes256-cts/3ADD

means everything reported by the KRB5_TRACE output looks fine and a ticket was successfully acquired, so you should see a host/svcdns.domain@DOMAIN ticket in klist.

I'm puzzled by mention of gssapi-keyex in

testuser@svcdns.domain: Permission denied (gssapi-keyex,gssapi-with-mic)

because OpenSSH for Windows 8.1/8.6 don't yet support gssapi-keyex (#15), it only does gssapi-with-mic so far.

Does sshd say in

$ journalctl _SYSTEMD_UNIT=ssh.service

why it denied permission? Perhaps the PAM stack doesn't like the user name provided?

What does klist show afterwards?

identical as before

Just to clarify: I had meant just klist to check what ticket the user got as a result of attempting ssh -K, not klist -kt to check the keytab file of the server.

The only thing I found in my AD record for the linux machine is that the "Computer name" is full capital letters (no FQDN) but the "DNS name" is all lower cases (with FQDN). Could this be an issue?

I don't think so. SSPI doesn't need any of these entries for SSH. The Linux server doesn't really need to be fully “domain joined” in a Windows sense for ssh -K, all it needs is an SPN and a keytab. As I said, we just use normal user entries for our Linux servers and that works as well, e.g.

$ ldapsearch -Y GSSAPI -Q -LLL -H ldap://adsrv.dc.cl.cam.ac.uk -b DC=dc,DC=cl,DC=cam,DC=ac,DC=uk cn=hostDirac
dn: CN=hostDirac,OU=krb5ServicePrincipals,DC=dc,DC=cl,DC=cam,DC=ac,DC=uk
objectClass: top
objectClass: person
objectClass: organizationalPerson
objectClass: user
cn: hostDirac
description: host/dirac.cl.cam.ac.uk
distinguishedName: CN=hostDirac,OU=krb5ServicePrincipals,DC=dc,DC=cl,DC=cam,DC=ac,DC=uk
instanceType: 4
whenCreated: 20160923125142.0Z
whenChanged: 20200723083126.0Z
displayName: hostDirac
uSNCreated: 67775
uSNChanged: 18297641
name: hostDirac
objectGUID:: wChZHsBwPUqBUH8iR6q4kQ== # c0 28 59 1e c0 70 3d 4a 81 50 7f 22 47 aa b8 91
userAccountControl: 544 # PASSWD_NOTREQD(32) | NORMAL_ACCOUNT(512)
codePage: 0
countryCode: 0
primaryGroupID: 513
objectSid:: AQUAAAAAAAUVAAAAVHhoalqY1Is9OO6uFg8AAA== # S-1-5-5-21-1785231444-2345965658-2934847549-3862
sAMAccountName: hostDirac
sAMAccountType: 805306368
userPrincipalName: host/dirac.cl.cam.ac.uk@DC.CL.CAM.AC.UK
servicePrincipalName: host/dirac.cl.cam.ac.uk
objectCategory: CN=Person,CN=Schema,CN=Configuration,DC=dc,DC=cl,DC=cam,DC=ac,DC=uk
dSCorePropagationData: 20170208114138.0Z
dSCorePropagationData: 16010101000001.0Z
msDS-SupportedEncryptionTypes: 24 # AES128-CTS-HMAC-SHA1-96(8) | AES256-CTS-HMAC-SHA1-96(16)

I think only the servicePrincipalName and msDS-SupportedEncryptionTypes attributes really matter here for Kerberos use by SSH: one to find the entry (and associated password/keytab), and the other to control which encryption types are allowed (make sure the AES ones are enabled).

Can you use ldapsearch as above to query your domain controller via GSSAPI authentication? You will need

sudo apt-get install krb5-user ldap-utils libsasl2-modules-gssapi-mit

On any Linux machine (that can contact port 88 of the domain controller), can you get as a normal user with
$ kvno host/svrdns.domain
$ klist
a ticket for principal host/svrdns.domain?

If not, does
$ kvno host/svrdns.domain@DOMAIN
$ klist
work?

The kvno command command allows you to test getting a ticket, without actually using it. I think on Windows the klist get option, as in
C:\> klist get host/svrdns.domain
C:\> klist
might do the same.

Yeah kvno works and klist get also works. I am able to see a ticket in both linux and windows machines.

Linux:

$ klist Ticket cache: FILE:/tmp/krb5cc_892039766_dnXXV1 Default principal: testuser@DOMAIN

Valid starting Expires Service principal 01/10/22 07:25:41 01/10/22 17:25:41 krbtgt/DOMAIN@DOMAIN renew until 01/17/22 07:25:41 01/10/22 07:29:31 01/10/22 17:25:41 host/svrdns.domain@DOMAIN renew until 01/17/22 07:25:41

Windows:

2> Client: testuser @ DOMAIN

    Server: host/svrdns.domain @ DOMAIN
    KerbTicket Encryption Type: AES-256-CTS-HMAC-SHA1-96
    Ticket Flags 0x40a10000 -> forwardable renewable pre_authent name_canonicalize
    Start Time: 1/10/2022 7:36:18 (local)
    End Time:   1/10/2022 17:36:18 (local)
    Renew Time: 1/16/2022 22:46:07 (local)
    Session Key Type: AES-256-CTS-HMAC-SHA1-96
    Cache Flags: 0
    Kdc Called: ****

Does sshd say in
$ journalctl _SYSTEMD_UNIT=ssh.service
why it denied permission? Perhaps the PAM stack doesn't like the user name provided?

Somehow I don't see any log in the journalctl about the login attempt...

What does klist show afterwards?

identical as before

Just to clarify: I had meant just klist to check what ticket the user got as a result of attempting ssh -K, not klist -kt to check the keytab file of the server.

I see. But I think the service ticket for the host seems to show up without any problem. Just couldn't authenticate further.

On the AD side, we didn't turn on the encryption for the service account we are using to connect since it isn't on by default.

What does “turn on encryption” mean exactly? (What exact GUI control or LDAP attribute are you talking about?)

On the server machine we also didn't turn on "Trust this computer for delegation" option. Would you recommend to retry with these two options turned on?

If you want to delegate a ticket, for example because you want from your SSH Linux server to be able to use other Kerberos-authenticated services (e.g., an SMB or NFS file server), then you will need to enable delegation. However, whether delegation is enabled or not for that server should not affect the success of authentication. It should only affect if you have a forwarded ticket on the Linux server after you successfully authenticated.

I still suspect that gssapi-with-mic authentication actually works for you and permission is denied after successful Kerberos authentication, because something else didn't like the outcome of Kerberos authentication (e.g. the user name).

Can you run sshd on the Linux server in foreground debug mode, as in

# KRB5_TRACE=/tmp/ssh-krb5trace /sbin/sshd -D -d -p 2222

and then connect to port 2222 with ssh -K -p 2222 ...?

If I do this, I see gssapi-with-mic success reported by sshd with lines such as

Authorized to mgkuhn, krb5 principal mgkuhn@DOMAIN (krb5_kuserok)
[...]
Accepted gssapi-with-mic for mgkuhn from 198.51.100.21 port 35620 ssh2: mgkuhn@DOMAIN

Also have a look at the Kerberos library debug output in /tmp/ssh-krb5trace for clues. (Somehow KRB5_TRACE=/dev/stdout didn't work for me, possibly because some child process used has stdout closed?) I get there:

Decrypted AP-REQ with server principal host/myserver.fqdn@DOMAIN: aes256-cts/949F
AP-REQ ticket: mgkuhn@DOMAIN -> host/myserver.fqdn@DOMAIN, session key aes256-cts/FACA
Negotiated enctype based on authenticator: aes256-cts
Authenticator contains subkey: aes256-cts/05FF
Creating AP-REP, time 1641831034.673366, subkey aes256-cts/3A3E, seqnum 53985744

Can you run sshd on the Linux server in foreground debug mode, as in
# KRB5_TRACE=/tmp/ssh-krb5trace /sbin/sshd -D -d -p 2222
and then connect to port 2222 with ssh -K -p 2222 ...?

If I do this, I see gssapi-with-mic success reported by sshd with lines such as
Authorized to mgkuhn, krb5 principal mgkuhn@DOMAIN (krb5_kuserok)
[...]
Accepted gssapi-with-mic for mgkuhn from 198.51.100.21 port 35620 ssh2: mgkuhn@DOMAIN

Yeah all the debug output I produced was using sshd on port 2222.

Also have a look at the Kerberos library debug output in /tmp/ssh-krb5trace for clues. (Somehow KRB5_TRACE=/dev/stdout didn't work for me, possibly because some child process used has stdout closed?) I get there:
Decrypted AP-REQ with server principal host/myserver.fqdn@DOMAIN: aes256-cts/949F
AP-REQ ticket: mgkuhn@DOMAIN -> host/myserver.fqdn@DOMAIN, session key aes256-cts/FACA
Negotiated enctype based on authenticator: aes256-cts
Authenticator contains subkey: aes256-cts/05FF
Creating AP-REP, time 1641831034.673366, subkey aes256-cts/3A3E, seqnum 53985744

I thihnk the stdout worked for me and I believe I have posted the output of the KRB5_TRACE above.

$ ldapsearch -Y GSSAPI -Q -LLL -H ldap://domain -b DC=***,DC=com cn=svrdns
dn: CN=SRVDNS,OU=Linux,OU=***,OU=Servers,DC=***,DC=com
objectClass: top
objectClass: person
objectClass: organizationalPerson
objectClass: user
objectClass: computer
cn: SRVDNS
distinguishedName: CN=SRVDNS,OU=Linux,OU=***,OU=Servers,DC=***,DC=com
instanceType: 4
whenCreated: 20210708173713.0Z
whenChanged: 20220105034836.0Z
uSNCreated: 17092343
memberOf: CN=swd-r-defenderatponly,OU=Required,OU=Software,OU=Groups,DC=***,DC=com
uSNChanged: 21768232
name: SVRDNS
objectGUID:: ****
userAccountControl: 69632
codePage: 0
countryCode: 0
localPolicyFlags: 0
pwdLastSet: 132842534891110482
primaryGroupID: 515
objectSid:: AQUAAAAAAAUVAAAA+3UDDaFR5EKeXbdraxcBAA==
accountExpires: 9223372036854775807
sAMAccountName: SVRDNS$
sAMAccountType: 805306369
operatingSystem: pc-linux-gnu
dNSHostName: svrdns.domain
servicePrincipalName: RestrictedKrbHost/svrdns.domain
servicePrincipalName: RestrictedKrbHost/SVRDNS
servicePrincipalName: host/svrdns.domain
servicePrincipalName: host/SVRDNS
objectCategory: CN=Computer,CN=Schema,CN=Configuration,DC=****,DC=com
isCriticalSystemObject: FALSE
dSCorePropagationData: 16010101000000.0Z
lastLogonTimestamp: 132858277309780824
msDS-SupportedEncryptionTypes: 28

This is the account for the server though. I don't know why but I couldn't search for the "testuser" account I used to connect in LDAP.

On the AD side, we didn't turn on the encryption for the service account we are using to connect since it isn't on by default.

What does “turn on encryption” mean exactly? (What exact GUI control or LDAP attribute are you talking about?)

The user account does not have encryption turned on:

The only thing I found in my AD record for the linux machine is that the "Computer name" is full capital letters (no FQDN) but the "DNS name" is all lower cases (with FQDN). Could this be an issue?

I don't think so. SSPI doesn't need any of these entries for SSH. The Linux server doesn't really need to be fully “domain joined” in a Windows sense for ssh -K, all it needs is an SPN and a keytab. As I said, we just use normal user entries for our Linux servers and that works as well, e.g.
$ ldapsearch -Y GSSAPI -Q -LLL -H ldap://adsrv.dc.cl.cam.ac.uk -b DC=dc,DC=cl,DC=cam,DC=ac,DC=uk cn=hostDirac
dn: CN=hostDirac,OU=krb5ServicePrincipals,DC=dc,DC=cl,DC=cam,DC=ac,DC=uk
objectClass: top
objectClass: person
objectClass: organizationalPerson
objectClass: user
cn: hostDirac
description: host/dirac.cl.cam.ac.uk
distinguishedName: CN=hostDirac,OU=krb5ServicePrincipals,DC=dc,DC=cl,DC=cam,DC=ac,DC=uk
instanceType: 4
whenCreated: 20160923125142.0Z
whenChanged: 20200723083126.0Z
displayName: hostDirac
uSNCreated: 67775
uSNChanged: 18297641
name: hostDirac
objectGUID:: wChZHsBwPUqBUH8iR6q4kQ== # c0 28 59 1e c0 70 3d 4a 81 50 7f 22 47 aa b8 91
userAccountControl: 544 # PASSWD_NOTREQD(32) | NORMAL_ACCOUNT(512)
codePage: 0
countryCode: 0
primaryGroupID: 513
objectSid:: AQUAAAAAAAUVAAAAVHhoalqY1Is9OO6uFg8AAA== # S-1-5-5-21-1785231444-2345965658-2934847549-3862
sAMAccountName: hostDirac
sAMAccountType: 805306368
userPrincipalName: host/dirac.cl.cam.ac.uk@DC.CL.CAM.AC.UK
servicePrincipalName: host/dirac.cl.cam.ac.uk
objectCategory: CN=Person,CN=Schema,CN=Configuration,DC=dc,DC=cl,DC=cam,DC=ac,DC=uk
dSCorePropagationData: 20170208114138.0Z
dSCorePropagationData: 16010101000001.0Z
msDS-SupportedEncryptionTypes: 24 # AES128-CTS-HMAC-SHA1-96(8) | AES256-CTS-HMAC-SHA1-96(16)
I think only the servicePrincipalName and msDS-SupportedEncryptionTypes attributes really matter here for Kerberos use by SSH: one to find the entry (and associated password/keytab), and the other to control which encryption types are allowed (make sure the AES ones are enabled).

Can you use ldapsearch as above to query your domain controller via GSSAPI authentication? You will need
sudo apt-get install krb5-user ldap-utils libsasl2-modules-gssapi-mit

So here looks like your 'hostDirac' account is actually a user account as oppose to a computer account in my case? Do I need to create a user account to represent the machine? I think our convention is to use computer account for machine and user account only for real user + service accounts.

Do I need to create a user account to represent the machine?

I don't think it makes a difference here, but generally, Active Directory computer accounts are intended for Windows computers, not for other computers that also use Kerberos. A Windows computer will manage its own object in Active Directory via LDAP, another Kerberos-using computer may not. (But more modern things like Samba and sssd may be much closer to Windows in that respect than traditional Unix machines with manually-configured MIT Kerberos.) The ktpass documentation says “to create a user account for a service on a computer that is not running the Windows operating system”, so that's what we did. (It also says “You cannot map multiple service instances to the same user account.”, but I suspect that's just a limitation of the ktpass CLI, not a limitation of Active Directory, because computer account objects usually have lots of servicePrincipalName attributes, not just one.)

Further comparing our AD LDAP entries for the servers, I also note the following differences:

I have

userAccountControl: 544 # PASSWD_NOTREQD(32) | NORMAL_ACCOUNT(512)

whereas you have

userAccountControl: 69632 # WORKSTATION_TRUST_ACCOUNT(4096) | DONT_EXPIRE_PASSWORD(65536)

I don't know what exactly these mean and if these account-control bits matter here.

I see that you haven't enabled AES in the GUI, but nevertheless it seems enabled in your

msDS-SupportedEncryptionTypes: 28 # RC4-HMAC(4) | AES128-CTS-HMAC-SHA1-96(8) | AES256-CTS-HMAC-SHA1-96(16)

There is also mention at ktpass of three “principal types” selected with /ptype and again I don't know if these matter here and what effect they have. (The documentation doesn't say.)

Also check if the LDAP object for your testuser has the two AES encryption types (bit values 8 + 16 = 24 set in msDS-SupportedEncryptionTypes) enabled, i.e. make sure those AES boxes are ticked in the GUI for your users and servers.

Background: Windows NT only supported DES encryption types and Windows 2000 only added the RC4 encryption type, but all these encryption types that are now commonly considered insecure (DES because it has 56-bit keys and RC4 because it uses the same key as NTLM). While Windows Vista finally added support for two AES encryption types, sadly nobody at Microsoft didn't switch it on by default for new users in Active Directory. At first, they were probably worried that these users then couldn't log into Windows NT/2000/XP machines (backwards compatibility), but Windows XP is long dead now and so one can now safely assume in 2022 that all Kerberos implementations out there have supported AES for more than a decade and therefore it really should be enabled by default for all users. So yes, please do tick those AES boxes.

I have long advocated to switch off RC4 and switch on AES. Very simple. Looking at the 2020 blog post Decrypting the Selection of Supported Kerberos Encryption Types I can see that some people at Microsoft are finally also (very slowly) coming round to that view, but are forever having backwards compatibility concerns with technology from the previous century. There are many Kerberos implementations (most notably macOS) that never supported RC4, and many that have started disabling it.

So based on the information provided,

creating "user account" for server is what you have tested with but "computer account" should work
next config item to test is to set up AES for the testuser

I will give it a try with #2 and get back here. Although I am not very sure if our domain admin will approve this change.

Unfortunately even with the "AES encryption" turned on for the testuser, GSSAPI still failed from linux -> linux.

Failed gssapi-with-mic for testuser from 10.1.27.104 port 35886 ssh2
debug3: mm_ssh_gssapi_userok: user not authenticated [preauth]

I'm afraid I'm running out of ideas for advice that I can give this way, i.e. without being able to actually see the entire configuration and debug output myself.

Generally, when I was able to help with getting Kerberos to work, it usually came down to one or more of the following things to check, in case you want to start over from scratch once more:

Do both sides agree on the principal names and realm to be used? If not, this is usually due to some problem with DNS, lack of rDNS (if relied upon), hostname, default realm, use of numeric IP addresses or NAT.
Does the client in preauth use an encryption type that the KDC accepts?
Does the KDC pick in the ticket encryption types that both client and server will accept? Recall that Kerberos has no way of negotiating encryption types, i.e. each party must pick one encryption type out of the set that the other side can accept, or the connection attempt will fail.
Is Kerberos authentication actually working, but the user name provided by Kerberos to the login server is not being accepted there (e.g., because the realm/domain was included and not stripped off)?
Is ticket delegation enabled in Active Directory with “user/computer trusted for delegation” (if you want to delegate a ticket, not essential for just authentication)?
Is the keytab not installed correctly on a Linux server (correct key exported with ktpass, required encryption type(s) included, matching version number, correct keytab file permissions, correct keytab path)
Do other Kerberos applications work, even on the same system?
Does "kvno" get a ticket for the desired service?
What's in /etc/krb5.conf or $KRB5_CONFIG (e.g., leftovers from previous experiments)?
Is option GSSAPIAuthentication enabled for OpenSSH on both sides in ssh_config (or use ssh -K) and sshd_config?
Is the OpenSSH for Windows version used 7.9 or newer?
Is the problem limited to one computer or can it be reproduced on several?
On very old Active Directory servers: is domain functional level 2008 enabled (before that there was no AES support)?
Are the Active Directory user accounts involved active (e.g., no expired password, no password change for server since last keytab transfer, etc.)?

On the Unix/Linux side, Kerberos debugging is done via KRB5_TRACE (see "man kerberos"). The "klist" command shows what ticket was obtained by the user. The "klist -kt" command lists the keytab contents.

On Windows, SSPI debugging can be done via HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Lsa\Kerberos\Parameters REG_DWORD LogLevel=1. The klist command shows what ticket was obtained by the user.

(Regarding OpenSSH debug output, I usually find "ssh -v" enough, and "ssh -vvv" just clutters the output with irrelevant events.)

Wireshark can be used to look at Kerberos protocol packets to/from KDC (but is less useful for looking at GSSAPI exchanged running inside the encrypted SSH tunnel).

(I'll add more here in case I can think of anything else.)

@mgkuhn Thank you so much for your advice.

I think chances are I will put it on hold for now and revisit when I have time to try what you suggested from scratch.

To me everything looks fine on the Kerberos authentication. It reads to me like sshd is rejecting the client as unsuitable for logging into this local user’s account.

You mention several different names for the client; testuser@DOMAIN (the Kerberos client principal used to authenticate), domain\\testuser (a Microsoft specific login syntax), domain\\testuser@svcdns (another Microsoft invention), and testuser@svcdns.domain (apparently the target account). Unless you or sssd define a mapping somewhere, these names are not equivalent to the local account “testuser”.

As an interim measure, you can create a .k5login file, in testuser’s homedir, owned by testuser with permissions 400, with the content

    testuser@DOMAIN

This allows the specified principal(s) to login as the target account (unless suppressed by sshd config). But a better way is to define the mapping centrally. Normally, unless no other configuration is made, sshd assumes that xyz@DEFAULTREALM is entitled to logon as local account xyz, where the DEFAULTREALM is deduced from the krb5.conf file.

I would be interested to see the output of KRB5_TRACE from the server. So far you only provided the client side. sssd’s logs would also be informative.

Finally, I’m confused about the name of the host. Is it svcdns or SVRDNS? It can have multiple names if you like, but I would get it working before adding lots of aliases.

I was having the same issue with similar error logs trying to set up using SSH with a Linux server and a Windows client making use of Active Directory SSO, using a similar if not exact setup using realm, sssd, and samba. After some searching around, I found one thing which appeared to fix my case. Apparently, sss creates files in /var/lib/sss/pubconf/krb5.include.d/ that are intended to be included by /etc/krb5.conf. The only file which appeared to have any settings which I believe could have an effect is /var/lib/sss/pubconf/krb5.include.d/localauth_plugin.

[plugins]
 localauth = {
  module = sssd:/usr/lib/x86_64-linux-gnu/sssd/modules/sssd_krb5_localauth_plugin.so
 }

As soon as I added includedir /var/lib/sss/pubconf/krb5.include.d/ at the top of /etc/krb5.conf, restarted sssd and realmd, started sshd (as I had it turned off for debugging), and attempted to log in from the client as ssh -K server.domain, everything worked without issue.

This is brilliant, @TheJayMann's solution worked for me as well on Ubuntu 22.04 with sssd and sshd.

Just in case the admin wonders, I am no longer at the company where I reported this issue with. Feel free to close this issue.

I would love to see if @TheJayMann's solution works. Sounds like there are positive feedback here.

Cheers

@TheJayMann Do you recall where you find the solution?

As long ago as it was that I found it, I am not sure what led me to this discovery. It could have been looking at bug reports on different distributions, it could have been inspecting file names of the various packages using apt-file until finding something interesting, or it could have been various attempts at listing search results which others expressed similar issues and potential fixes. Searching the web for the full path of the directory to be included do return some interesting results, including a github issue for sssd, which itself is copied from a Red Had bugzilla bug report, which describe the problem and the solution very similar to my response. https://github.com/SSSD/sssd/issues/5893 If looking for the source, this is probably as good as anything else one could find.

@RobCrowston .k5login make it works for me. Thank you so much!

Reported to Canonical in September 2023 here: https://bugs.launchpad.net/ubuntu/+source/sssd/+bug/2037321 If you are affected you could maybe write a line there or even better if you bought support from canonical report it to them as a support case.

I was having the same issue with similar error logs trying to set up using SSH with a Linux server and a Windows client making use of Active Directory SSO, using a similar if not exact setup using realm, sssd, and samba. After some searching around, I found one thing which appeared to fix my case. Apparently, sss creates files in /var/lib/sss/pubconf/krb5.include.d/ that are intended to be included by /etc/krb5.conf. The only file which appeared to have any settings which I believe could have an effect is /var/lib/sss/pubconf/krb5.include.d/localauth_plugin.
[plugins]
 localauth = {
  module = sssd:/usr/lib/x86_64-linux-gnu/sssd/modules/sssd_krb5_localauth_plugin.so
 }
As soon as I added includedir /var/lib/sss/pubconf/krb5.include.d/ at the top of /etc/krb5.conf, restarted sssd and realmd, started sshd (as I had it turned off for debugging), and attempted to log in from the client as ssh -K server.domain, everything worked without issue.

Wow man @TheJayMann I've had this problem for days without solutions on Debian 12 and you came to the rescue.

In /etc/ssh/sshd_config, had to set GSSAPIStrictAcceptorCheck no, otherwise the ssh server will pop this error: No key table entry found matching host/hostname@ (missing realm after @), solution here.
Added includedir /var/lib/sss/pubconf/krb5.include.d/ as you did, and now it works like a charm. On a client I can just type ssh -K host and I'll get authenticated with my AD account (the one I used to start the SSH client) without typing a password.

Must apply both solutions otherwise it won't work.

PowerShell / Win32-OpenSSH

GSSAPI Authentication from Windows to Linux failed #1875

2> Client: testuser @ DOMAIN

2> Client: testuser @ DOMAIN