Rsync doesnt work with zstd compression and fails

[breisig]

I have been testing since Rsync 3.2.0 and even the latest rsync from git today. and it seems when running rsync with compression (zstd), it never seems to finished and errors with error code 10. I have zstd 1.4.5 installed on my system. Using the parameters below. (not full command)

sudo -E /usr/bin/rsync -az --zc zstd -e 'ssh -CA -p 22 -oStrictHostKeyChecking=no -oPasswordAuthentication=no -oBatchMode=yes -oUserKnownHostsFile=/dev/null' --rsync-path='sudo rsync' --progress -vv --force --ignore-errors --delete-excluded --delete --numeric-ids --no-specials --no-devices --timeout=3600 --prune-empty-dirs

It always fails with

rsync: [generator] write error: Broken pipe (32) rsync error: error in socket IO (code 10) at io.c(817) [generator=3.2.2pre3]

However, if I try to force the compression to zlib (--zc zlib) or disable compression, it seems to work. Not sure if by default it's using zstd between both ends.

[WayneD]

I haven't seen it fail yet, but will see if I can work up a reproducible test. Could you give --zc=lz4 a try and see if that is working for you? Also, what does the remote rsync respond to rsync --version?

[breisig]

@WayneD

Compiled it again from git. I don't have lz4

root@server tmp# rsync --version
rsync  version 3.2.2pre3  protocol version 31
Copyright (C) 1996-2020 by Andrew Tridgell, Wayne Davison, and others.
Web site: https://rsync.samba.org/
Capabilities:
    64-bit files, 64-bit inums, 64-bit timestamps, 64-bit long ints,
    socketpairs, hardlinks, symlinks, IPv6, atimes, batchfiles, inplace,
    append, ACLs, xattrs, optional protect-args, iconv, symtimes, prealloc
Optimizations:
    SIMD, asm, openssl-crypto
Checksum list:
    md5 md4 none
Compress list:
    zstd zlibx zlib none

rsync comes with ABSOLUTELY NO WARRANTY.  This is free software, and you
are welcome to redistribute it under certain conditions.  See the GNU
General Public Licence for details.

[WayneD]

Is that version output the local machine the remote machine or both? How many files are in your transfer? I've tested around 60K transferred files without issue, but that's just a few GB of data. It would also be good to know why the remote side of the transfer is actually dying (since the only error you reported is just the closed socket). You could try something like --rsync-path='sudo strace -o /tmp/out rsync' if that won't overflow /tmp (and feel free to periodically run cp /dev/null /tmp/out while you're waiting for it to die).