rsync: connection unexpectedly closed (0 bytes received so far) [Receiver]

doggy8088 commented 2 years ago

Troubleshooting steps https://github.com/PowerShell/Win32-OpenSSH/wiki/Troubleshooting-Steps

Configure OpenSSH Server

Invoke-WebRequest -Uri https://github.com/PowerShell/Win32-OpenSSH/releases/download/V8.6.0.0p1-Beta/OpenSSH-Win64.zip -OutFile "$env:TEMP/OpenSSH-Win64.zip"

Expand-Archive -LiteralPath "$env:TEMP/OpenSSH-Win64.zip" -DestinationPath "$env:TEMP\OpenSSH"

mkdir "$env:ProgramFiles\OpenSSH"
Copy-Item -Path "$env:TEMP\OpenSSH\OpenSSH-Win64\*" -Destination "$env:ProgramFiles\OpenSSH" -Recurse

cd "$env:ProgramFiles\OpenSSH"
powershell.exe -ExecutionPolicy Bypass -File install-sshd.ps1

Start-Service sshd

Set-Service -Name sshd -StartupType 'Automatic'

New-NetFirewallRule -Name sshd -DisplayName 'OpenSSH Server (sshd)' -Enabled True -Direction Inbound -Protocol TCP -Action Allow -LocalPort 22

New-ItemProperty -Path "HKLM:\SOFTWARE\OpenSSH" -Name DefaultShell -Value "C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe" -PropertyType String -Force

Install cwRsync

Invoke-WebRequest -Uri https://itefix.net/dl/free-software/cwrsync_6.2.4_x64_free.zip -OutFile "$env:TEMP/cwrsync_6.2.4_x64_free.zip"

Expand-Archive -LiteralPath "$env:TEMP\cwrsync_6.2.4_x64_free.zip" -DestinationPath "$env:TEMP\cwrsync"

mkdir "C:\cwrsync"
Copy-Item -Path "$env:TEMP\cwrsync\cwrsync_6.2.4_x64_free\*" -Destination "C:\cwrsync" -Recurse

# Append C:\cwrsync\bin to PATH system environment variable
Invoke-WebRequest -Uri "https://github.com/doggy8088/SetEnv/releases/download/1.0/SetEnv.exe" -OutFile "SetEnv.exe"
.\SetEnv.exe -a PATH %C:\cwrsync\bin

# Restart sshd
Restart-Service sshd

Create user1 user locally with password 123
```
net user user1 123 /add
```

Try to copy file using scp

echo 123 > test.txt
scp .\test.txt user1@localhost:.

Try to copy file using rsync

rsync -e 'ssh.exe' -avzr .\test.txt user1@localhost:.

This will fail.

rsync: connection unexpectedly closed (0 bytes received so far) [Receiver]
rsync error: error in rsync protocol data stream (code 12) at io.c(228) [Receiver=3.2.3]
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
rsync error: error in rsync protocol data stream (code 12) at io.c(228) [sender=3.2.3]

I tried to troubleshoot this problem.

rsync -e 'ssh.exe -v' -avzr .\test.txt user1@localhost:.

It shows:

OpenSSH_for_Windows_8.1p1, LibreSSL 3.0.2
debug1: Connecting to localhost [::1] port 22.
debug1: Connection established.
debug1: identity file C:\\Users\\WDAGUtilityAccount/.ssh/id_rsa type -1
debug1: identity file C:\\Users\\WDAGUtilityAccount/.ssh/id_rsa-cert type -1
debug1: identity file C:\\Users\\WDAGUtilityAccount/.ssh/id_dsa type -1
debug1: identity file C:\\Users\\WDAGUtilityAccount/.ssh/id_dsa-cert type -1
debug1: identity file C:\\Users\\WDAGUtilityAccount/.ssh/id_ecdsa type -1
debug1: identity file C:\\Users\\WDAGUtilityAccount/.ssh/id_ecdsa-cert type -1
debug1: identity file C:\\Users\\WDAGUtilityAccount/.ssh/id_ed25519 type -1
debug1: identity file C:\\Users\\WDAGUtilityAccount/.ssh/id_ed25519-cert type -1
debug1: identity file C:\\Users\\WDAGUtilityAccount/.ssh/id_xmss type -1
debug1: identity file C:\\Users\\WDAGUtilityAccount/.ssh/id_xmss-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_for_Windows_8.1
debug1: Remote protocol version 2.0, remote software version OpenSSH_for_Windows_8.6
debug1: match: OpenSSH_for_Windows_8.6 pat OpenSSH* compat 0x04000000
debug1: Authenticating to localhost:22 as 'user1'
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:ceWbZ99XvvQUHF8bsSdEvvQaHpMaE54eGir9SQ+3cwc
debug1: Host 'localhost' is known and matches the ECDSA host key.
debug1: Found key in C:\\Users\\WDAGUtilityAccount/.ssh/known_hosts:3
debug1: rekey out after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: rekey in after 134217728 blocks
debug1: pubkey_prepare: ssh_get_authentication_socket: No such file or directory
debug1: Will attempt key: C:\\Users\\WDAGUtilityAccount/.ssh/id_rsa
debug1: Will attempt key: C:\\Users\\WDAGUtilityAccount/.ssh/id_dsa
debug1: Will attempt key: C:\\Users\\WDAGUtilityAccount/.ssh/id_ecdsa
debug1: Will attempt key: C:\\Users\\WDAGUtilityAccount/.ssh/id_ed25519
debug1: Will attempt key: C:\\Users\\WDAGUtilityAccount/.ssh/id_xmss
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,sk-ssh-ed25519@openssh.com,ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ecdsa-sha2-nistp256@openssh.com,webauthn-sk-ecdsa-sha2-nistp256@openssh.com>
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,password,keyboard-interactive
debug1: Next authentication method: publickey
debug1: Trying private key: C:\\Users\\WDAGUtilityAccount/.ssh/id_rsa
debug1: Trying private key: C:\\Users\\WDAGUtilityAccount/.ssh/id_dsa
debug1: Trying private key: C:\\Users\\WDAGUtilityAccount/.ssh/id_ecdsa
debug1: Trying private key: C:\\Users\\WDAGUtilityAccount/.ssh/id_ed25519
debug1: Trying private key: C:\\Users\\WDAGUtilityAccount/.ssh/id_xmss
debug1: Next authentication method: keyboard-interactive
debug1: Authentications that can continue: publickey,password,keyboard-interactive
debug1: Next authentication method: password
debug1: read_passphrase: can't open /dev/tty: No such file or directory
user1@localhost's password:
debug1: Authentication succeeded (password).
Authenticated to localhost ([::1]:22).
debug1: channel 0: new [client-session]
debug1: Requesting no-more-sessions@openssh.com
debug1: Entering interactive session.
debug1: pledge: network
debug1: client_input_global_request: rtype hostkeys-00@openssh.com want_reply 0
debug1: Sending command: rsync --server -vlogDtprze.iLsfxCIvu . .
rsync: connection unexpectedly closed (0 bytes received so far) [Receiver]
rsync error: error in rsync protocol data stream (code 12) at io.c(228) [Receiver=3.2.3]
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
rsync error: error in rsync protocol data stream (code 12) at io.c(228) [sender=3.2.3]

I can't really understand what's the problem.

Change to use cwRsync's ssh.exe

rsync -e 'c:\cwrsync\bin\ssh.exe' -avzr .\test.txt user1@localhost:.

This will okay!

sending incremental file list
.\test.txt

sent 110 bytes  received 41 bytes  60.40 bytes/sec
total size is 12  speedup is 0.08

Try to show ssh logs:

rsync -e 'c:\cwrsync\bin\ssh.exe -v' -avzr .\test.txt user1@localhost:.

The result:

OpenSSH_8.8p1, LibreSSL 3.3.3
debug1: Authenticator provider $SSH_SK_PROVIDER did not resolve; disabling
debug1: Connecting to localhost [::1] port 22.
debug1: Connection established.
debug1: identity file /cygdrive/c/Users/WDAGUtilityAccount/.ssh/id_rsa type -1
debug1: identity file /cygdrive/c/Users/WDAGUtilityAccount/.ssh/id_rsa-cert type -1
debug1: identity file /cygdrive/c/Users/WDAGUtilityAccount/.ssh/id_dsa type -1
debug1: identity file /cygdrive/c/Users/WDAGUtilityAccount/.ssh/id_dsa-cert type -1
debug1: identity file /cygdrive/c/Users/WDAGUtilityAccount/.ssh/id_ecdsa type -1
debug1: identity file /cygdrive/c/Users/WDAGUtilityAccount/.ssh/id_ecdsa-cert type -1
debug1: identity file /cygdrive/c/Users/WDAGUtilityAccount/.ssh/id_ecdsa_sk type -1
debug1: identity file /cygdrive/c/Users/WDAGUtilityAccount/.ssh/id_ecdsa_sk-cert type -1
debug1: identity file /cygdrive/c/Users/WDAGUtilityAccount/.ssh/id_ed25519 type -1
debug1: identity file /cygdrive/c/Users/WDAGUtilityAccount/.ssh/id_ed25519-cert type -1
debug1: identity file /cygdrive/c/Users/WDAGUtilityAccount/.ssh/id_ed25519_sk type -1
debug1: identity file /cygdrive/c/Users/WDAGUtilityAccount/.ssh/id_ed25519_sk-cert type -1
debug1: identity file /cygdrive/c/Users/WDAGUtilityAccount/.ssh/id_xmss type -1
debug1: identity file /cygdrive/c/Users/WDAGUtilityAccount/.ssh/id_xmss-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_8.8
debug1: Remote protocol version 2.0, remote software version OpenSSH_for_Windows_8.6
debug1: compat_banner: match: OpenSSH_for_Windows_8.6 pat OpenSSH* compat 0x04000000
debug1: Authenticating to localhost:22 as 'user1'
debug1: load_hostkeys: fopen /cygdrive/c/Users/WDAGUtilityAccount/.ssh/known_hosts2: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh_known_hosts2: No such file or directory
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: ssh-ed25519
debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: SSH2_MSG_KEX_ECDH_REPLY received
debug1: Server host key: ssh-ed25519 SHA256:TVkVEMTOVm8acWX4VFrviwD9t4FeFxN5qILJjjYZiIQ
debug1: load_hostkeys: fopen /cygdrive/c/Users/WDAGUtilityAccount/.ssh/known_hosts2: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh_known_hosts2: No such file or directory
debug1: Host 'localhost' is known and matches the ED25519 host key.
debug1: Found key in /cygdrive/c/Users/WDAGUtilityAccount/.ssh/known_hosts:1
debug1: rekey out after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: rekey in after 134217728 blocks
debug1: Will attempt key: /cygdrive/c/Users/WDAGUtilityAccount/.ssh/id_rsa
debug1: Will attempt key: /cygdrive/c/Users/WDAGUtilityAccount/.ssh/id_dsa
debug1: Will attempt key: /cygdrive/c/Users/WDAGUtilityAccount/.ssh/id_ecdsa
debug1: Will attempt key: /cygdrive/c/Users/WDAGUtilityAccount/.ssh/id_ecdsa_sk
debug1: Will attempt key: /cygdrive/c/Users/WDAGUtilityAccount/.ssh/id_ed25519
debug1: Will attempt key: /cygdrive/c/Users/WDAGUtilityAccount/.ssh/id_ed25519_sk
debug1: Will attempt key: /cygdrive/c/Users/WDAGUtilityAccount/.ssh/id_xmss
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,sk-ssh-ed25519@openssh.com,ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ecdsa-sha2-nistp256@openssh.com,webauthn-sk-ecdsa-sha2-nistp256@openssh.com>
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,password,keyboard-interactive
debug1: Next authentication method: publickey
debug1: Trying private key: /cygdrive/c/Users/WDAGUtilityAccount/.ssh/id_rsa
debug1: Trying private key: /cygdrive/c/Users/WDAGUtilityAccount/.ssh/id_dsa
debug1: Trying private key: /cygdrive/c/Users/WDAGUtilityAccount/.ssh/id_ecdsa
debug1: Trying private key: /cygdrive/c/Users/WDAGUtilityAccount/.ssh/id_ecdsa_sk
debug1: Trying private key: /cygdrive/c/Users/WDAGUtilityAccount/.ssh/id_ed25519
debug1: Trying private key: /cygdrive/c/Users/WDAGUtilityAccount/.ssh/id_ed25519_sk
debug1: Trying private key: /cygdrive/c/Users/WDAGUtilityAccount/.ssh/id_xmss
debug1: Next authentication method: keyboard-interactive
debug1: Authentications that can continue: publickey,password,keyboard-interactive
debug1: Next authentication method: password
user1@localhost's password:
Authenticated to localhost ([::1]:22) using "password".
debug1: channel 0: new [client-session]
debug1: Requesting no-more-sessions@openssh.com
debug1: Entering interactive session.
debug1: pledge: filesystem full
debug1: client_input_global_request: rtype hostkeys-00@openssh.com want_reply 0
debug1: client_input_hostkeys: searching /cygdrive/c/Users/WDAGUtilityAccount/.ssh/known_hosts for localhost / (none)
debug1: client_input_hostkeys: searching /cygdrive/c/Users/WDAGUtilityAccount/.ssh/known_hosts2 for localhost / (none)
debug1: client_input_hostkeys: hostkeys file /cygdrive/c/Users/WDAGUtilityAccount/.ssh/known_hosts2 does not exist
debug1: client_input_hostkeys: no new or deprecated keys from server
debug1: Sending command: rsync --server -vlogDtprze.iLsfxCIvu . .
sending incremental file list
rsync: [sender] link_stat "/cygdrive/c/Users/WDAGUtilityAccount/.\test.txt" failed: No such file or directory (2)
debug1: client_input_channel_req: channel 0 rtype exit-status reply 0
debug1: client_input_channel_req: channel 0 rtype eow@openssh.com reply 0
debug1: channel 0: free: client-session, nchannels 1
Transferred: sent 2164, received 2704 bytes, in 1.6 seconds
Bytes per second: sent 1328.0, received 1659.4
debug1: Exit status 0

sent 19 bytes  received 12 bytes  8.86 bytes/sec
total size is 0  speedup is 0.00
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1330) [sender=3.2.3]

Terminal issue? please go through wiki https://github.com/PowerShell/Win32-OpenSSH/wiki/TTY-PTY-support-in-Windows-OpenSSH

Please answer the following

"OpenSSH for Windows" version ((Get-Item (Get-Command sshd).Source).VersionInfo.FileVersion)

 8.6.0.0

Server OperatingSystem ((Get-ItemProperty "HKLM:\SOFTWARE\Microsoft\Windows nt\CurrentVersion\" -Name ProductName).ProductName)

Windows 10 Enterprise

Client OperatingSystem

Same machine

Windows 10 Enterprise

What is failing

Show in Troubleshooting steps Step 5.

Expected output

Show in Troubleshooting steps Step 6.

I expected the Win32-OpenSSH's ssh.exe should able to use with any program.

Actual output

Show in Troubleshooting steps Step 5.

mgkuhn commented 2 years ago

A possibly related question: how can one control in OpenSSH for Windows whether sshd performs input and output with a command that it invokes either

directly by connecting to its stdin, stdout, stderr via low-level I/O
indirectly by calling ssh-shellhost.exe/ConPTY to communicate with a console application via console I/O/ConsoleAPI

That distinction between binary-transparent low-level I/O and cursor-control-oriented console I/O in Windows does not exist under Unix, and therefore I assume there must be some selection mechanism to tell OpenSSH for Windows, which of these to use. I have a suspicion that distinction may be involved in why people have problems using OpenSSH for Windows as a transport tunnel with Unix applications such as rsync, git and subversion, which all pass compressed binary files via stdio to the other side, i.e. are definitely not using SSH as any kind of console application (with UTF-8, ANSI codes and line feeds). Because the ConPTY route will translate/rewrite all ANSI control sequences and is therefore hardly suited as a binary-transparent channel for compressed file transfer, which will only work via Unix-style pipes to redirected stdin/stdout via low-level _read() and _write(), right?

This is something that I never found well explained in the OpenSSH for Windows documentation, and as this distinction doesn't exist in Unix, the upstream OpenSSH documentation won't be able to help here either.

wangdong2023 commented 1 year ago

Met the exact same problem. Using cmd also doesn't help.

mgkuhn commented 1 year ago

Does this rsync: connection unexpectedly closed problem still occur with version v9.2.2.0p1 Beta? https://github.com/PowerShell/Win32-OpenSSH/releases/tag/v9.2.2.0p1-Beta Because that release fixed the long-standing bug #2012 that had caused a somewhat similar-looking early-EOF problem when OpenSSH for Windows is used with git fetch.

virzak commented 1 year ago

It is still there with v9.2.2.0p1-Beta

rsync: connection unexpectedly closed (0 bytes received so far) [Receiver]
rsync error: error in rsync protocol data stream (code 12) at io.c(231) [Receiver=3.2.7]
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
rsync error: error in rsync protocol data stream (code 12) at io.c(231) [sender=3.2.7]

ShenHongFei commented 1 year ago

Use msys2's ssh.exe to transfer normally. Win32-OpenSSH here keeps reporting errors.

rsync.exe -e E:/sdk/msys64/usr/bin/ssh.exe

PowerShell / Win32-OpenSSH

rsync: connection unexpectedly closed (0 bytes received so far) [Receiver] #1869