SSH client for Win10 hangs sometimes

[ngugcx]

Troubleshooting steps https://github.com/PowerShell/Win32-OpenSSH/wiki/Troubleshooting-Steps

Terminal issue? please go through wiki https://github.com/PowerShell/Win32-OpenSSH/wiki/TTY-PTY-support-in-Windows-OpenSSH

Please answer the following

"OpenSSH for Windows" version ((Get-Item (Get-Command sshd).Source).VersionInfo.FileVersion)

Server OperatingSystem ((Get-ItemProperty "HKLM:\SOFTWARE\Microsoft\Windows nt\CurrentVersion\" -Name ProductName).ProductName)

Client OperatingSystem Win10

What is failing Sometimes SSH get hung when connecting to Ubuntu 16.04 64bit. ssh -V OpenSSH_for_Windows_7.6p1, LibreSSL 2.6.4

I got the debug trace with -vvv: ssh.txt

Expected output don't hang

Actual output hang

[alexv1n]

I'm having a similar issue. Executing ssh.exe user@ip.address -- echo -n "" just hangs sometimes.

When it does hang, it happens at syncio_close() (in contrib\win32\win32compat\termio.c) at line 263 (WaitForSingleObject):

    if (pio->read_details.pending) {
        /*
        Terminate the read thread at the below situations:
        1. For console - the read thread is blocked by the while loop on raw mode
        2. Function ReadFile on Win7 machine dees not return when no content to read in non-interactive mode.
        */
        if (FILETYPE(pio) == FILE_TYPE_CHAR && (IsWin7OrLess() || in_raw_mode))
            TerminateThread(pio->read_overlapped.hEvent, 0);
        else
            WaitForSingleObject(pio->read_overlapped.hEvent, INFINITE);
    }

Sometimes hitting Enter in terminal unblocks ssh.exe and it exits successfully.

Also, if I run ssh user@server -- echo -n "" <NUL it blocks 100% of the time.

I have tried code from https://github.com/PowerShell/Win32-OpenSSH/issues/1338 but it doesn't fix the issue. This seems like a different bug, probably closely related to https://github.com/PowerShell/Win32-OpenSSH/issues/874 but it seems the issue is not confined to Win7 only. It happens on Win10 as well.

[manojampalam]

@alexv1n can you try out patched binaries from @nomorefood in this thread - https://github.com/PowerShell/Win32-OpenSSH/issues/1338

You would just need the updated version of ssh.exe.

[alexv1n]

I did say that I tried that and it didn't help

[Tyriar]

I'm seeing this as well, sometimes ssh <host> uname -sm hangs:

The window has been like this for 10+ minutes:

Pressing enter after that long:

Version info:

C:\Users\daimms.REDMOND>ssh -V
OpenSSH_for_Windows_7.7p1, LibreSSL 2.6.5

Windows 10 1903 18875.1000 (Insiders)

[bobvandevijver]

Seems this might be a low-level Windows issue, as I'm hitting something similar with kitty: https://github.com/cyd01/KiTTY/issues/22

[NoMoreFood]

@Tyriar Can you please try the version 8.0 binaries (and verify using ssh -V that you're actually using version 8.0 binaries, as you did with 7.7). Thanks!

[Tyriar]

Still hanging sometimes:

[NoMoreFood]

@Tyriar, alright I'll try to take a look and see if I can repro it. What's the purpose of wrapping the execution in a command prompt?

[Tyriar]

@NoMoreFood it might not make a difference but that's how we hit the bug initially

[AHelper]

FYI, I have been getting hangs on OpenSSH_for_Windows_7.9p1, LibreSSL 2.6.5. I pulled the latest and still get hangs:

PS C:\Users\user\Downloads\OpenSSH-Win64> .\ssh.exe -V
OpenSSH_for_Windows_8.0p1, LibreSSL 2.6.5
PS C:\Users\user\Downloads\OpenSSH-Win64> .\ssh.exe -vvv -E log2.txt -i .\id_rsa test@10.9.73.34 date
Tue Jul  9 18:50:29 UTC 2019
PS C:\Users\user\Downloads\OpenSSH-Win64> .\ssh.exe -vvvvvv -E log3.txt -i .\id_rsa test@10.9.73.34 date
                <<< Hung, enter pressed
Tue Jul  9 18:39:21 UTC 2019
PS C:\Users\user\Downloads\OpenSSH-Win64> .\ssh.exe -vvv -E log4.txt -i .\id_rsa test@10.9.73.34 date
Tue Jul  9 18:50:29 UTC 2019
                <<< Hung, enter pressed
PS C:\Users\user\Downloads\OpenSSH-Win64>

I find that running with -vvv alone masks the problem. Logging to file with -E allows the hang again. I annotated the logs where they stopped at the time they hung.

Now, this is about a 1-in-10 chance. I get different odds (more frequent) against an OpenSSH_7.9p1, OpenSSL 1.1.1b 26 Feb 2019 (Alpine 3.9) server.

Windows 10 Enterprise 1709 16299.1217 ssh: OpenSSH_for_Windows_8.0p1, LibreSSL 2.6.5 sshd: OpenSSH_7.4p1 Debian-10+deb9u6, OpenSSL 1.0.2s 28 May 2019

log3.txt log4.txt

Let me know if there's anything I should test out.

[asfarley]

I'm unsure if I'm hitting this issue or something else; when I try to ssh into my local Gitea server, the ssh command just hangs indefinitely.

In my case, this does not appear to be intermittent. I'm also able to connect to the server using Putty, so the server itself is alive and responding on the expected port.

ssh git@10.100.10.24 -p 22000 -i C:\Users\alex\.ssh\id_rsa

The Powershell terminal just sits there, unresponsive to Ctrl-C.

Does this sound like the same issue or something else? I'm willing to dive a bit deeper with Wireshark or other tools if that would help.

Version:

PS C:\Users\alexa\Desktop> ssh -V
OpenSSH_for_Windows_7.7p1, LibreSSL 2.6.5

[asfarley]

Since my previous comment, I found this post: https://apple.stackexchange.com/questions/35524/what-can-i-do-when-my-ssh-session-is-stuck

I have confirmed that typing the following sequence terminates the ssh connection: [ENTER]~.

Not sure what this means overall; I'm still unable to connect to my git server via SSH, but at least I'm able to close the terminal without having to hit 'X' on the window.

[highpost]

I had this problem and tracked it down to some lines in my .bashrc file. I'm running Ubuntu 20.04:

# If this is an xterm set the title to user@host:dir
case "$TERM" in
xterm*|rxvt*)
    PS1="\[\e]0;${debian_chroot:+($debian_chroot)}\u@\h: \w\a\]$PS1"
    ;;
*)
    ;;
esac

[11philip22]

I have the same problem when im trying to execute remote commands from a windows 2008 server. I have openssh 8.1p1 installed. It seems like command only shows output when i press enter. If i press enter and there is no output yet i have to press enter again. I think its an older Windows thing because when i tried it on Windows 10 or Linux it worked like it should have. And the ssh version i have installed on windows 10 is even lower (7.7p1)

[0if]

I have the same issue with Windows Server 2016 and Freesshd - RDP'ing into the server shows an error message 'powershell has stopped working'

[micky0867]

Same here (OpenSSH_for_Windows_8.1p1, LibreSSL 2.9.2)

I'm using a scheduler sw (BMC Control-M) to run a bunch of jobs on different servers. A couple of jobs are running on windows and need to execute commands on linux hosts using ssh. Sometimes these jobs got stuck and it seems to happen in the ssh disconnect phase.

I'm able to reproduce this when running multiple ssh's (10) at the same time. I'm using pub/priv authentication and a simple ssh: "ssh user@host uname -a"

Currently I'm running a test with 10 sessions in parallel, restarting every 2 minutes. The scheduler informs me, when one of the job isn't finished within 1 minute.

Just a simple

ssh user@host uname -a

got stuck from time to time, sometimes with output from the uname cmd, sometimes not.

The sockets on both sides are still in state ESTABLISHED when this happens. When I then kill the ssh process on the linux side, the socket on windows changes to CLOSE_WAIT and stays there (maybe forever?), while the linux side first changes to FIN_WAIT2 and disappears after some seconds.

I then changed the command to

ssh -v user@host uname -a

with different outputs

...
debug1: Sending command: uname -a
debug1: client_input_channel_req: channel 0 rtype exit-status reply 0
debug1: client_input_channel_req: channel 0 rtype eow@openssh.com reply 0
HANGS

...
debug1: Sending command: uname -a
Linux rt.xyz.de 4.14.35-1902.305.4.el7uek.x86_64 #2 SMP Tue Aug 4 14:17:05 PDT 2020 x86_64 x86_64 x86_64 GNU/Linux
debug1: client_input_channel_req: channel 0 rtype exit-status reply 0
debug1: client_input_channel_req: channel 0 rtype eow@openssh.com reply 0
HANGS

I changed sshd_config on the server side to

ClientAliveInterval 10
ClientAliveCountMax 3

sshd detects the hanging client and disconnects it, but the client process itself doesn't terminate (as expected).

When I change the command to

ssh -vvv user@host uname -a

it never hangs (as already mentioned by AHelper)

BTW: Before I found openssh, I used putty (plink) for this script. plink is a pain, but at least, it never got stuck

[ngugcx]

BTW: Before I found openssh, I used putty (plink) for this script. plink is a pain, but at least, it never got stuck

I'm preparing to use plink. Why is it a pain?

[micky0867]

I'm preparing to use plink. Why is it a pain?

I never found an option similar to StrictHostKeyChecking off

Yes, I know that this migth be an security issue, but I'm old enough to make my own decisions. And I have a need to connect to a round-robin cluster address.

plink explicitly needs parameters like -batch (Hey! Why do we have putty AND plink?) and there seems to be no default private key location, except the session is already configured in putty at least that it honors pageant

So for me, this is not the best tool to do things automagically in background.

In ssh I have my %HOMEDRIVE%%HOMEPATH%.ssh\id_rsa and %HOMEDRIVE%%HOMEPATH%.ssh\config and everything is fine.

BUT, when connecting interactively to sshd on Windows: Using ssh.exe, the screen doesn't scroll. Using plink.exe some output is garbled, but at least scrolling works Using putty.exe, everything is fine Using ssh from Linux, everything is fine

[farblos]

My 2 cents, based on

> ssh.exe -V
OpenSSH_for_Windows_7.7p1, LibreSSL 2.6.5

and

> .\tmp\ssh.exe -V
OpenSSH_for_Windows_8.6p1, LibreSSL 2.6.5

all done to the same Linux host over a low latency (1ms) connection:

Hangs almost always:

ssh.exe <host> "exit 0"

Hangs in 10% of cases:

ssh.exe <host> "echo -n ' '; exit 0"

Hangs in 4% (?) of cases when also writing to STDERR:

ssh.exe <host> "echo -n ' '; echo -n ' ' 1>&2; exit 0"

Hangs even less often (?) and does not add visible output:

ssh.exe <host> "echo -n $'\r'; echo -n $'\r' 1>&2; exit 0"

Hanged once in 500 subsequent tries:

ssh.exe <host> "echo -n $'\r'; echo -n $'\r' 1>&2; sleep 0.5; exit 0"

Using the last as a work-around in a wrapping powershell function.

[triatic]

Same, confirmed with both the OpenSSH_for_Windows_8.1p1 native Win10 build and the latest OpenSSH_for_Windows_9.5p1 release.

When executing a command such as below, around 20% of the time it doesn't return:

ssh user@server some_remote_command

The remote command can be very simple and it still periodically hangs. It seems like the pause is for some kind of keyboard input even though nothing in the remote command would want it, since pressing enter resumes processing. But running ssh in this manner is likely commonly used unattended, so this workaround is not going to be viable in many cases. That said, I only use this form in attended scripts.