Can cdc_rsync be used as a local Windows rsync tool?

[prarsena]

I saw the following in another issue thread :

adding support for Windows to Windows cdc_rsync (including local syncs, i.e. not over network), see https://github.com/google/cdc-file-transfer/compare/main...sync_windows

Between the speed improvements you highlight from this tool and the fact that Windows doesn't have native support for rsync, I would love to use this tool to make backups for local drives.

But I wasn't able to understand the status of that work. Is the current status of the sync_windows (or win_support) branch at the point to support local Win rsync?

If the support comes, would it work like similarly to the following format? cdc_rsync E:\ F:\MyDrive\ -rv

[ljusten]

The status is, the code is in review. I expect it to land next week. My reviewer is currently on vacation.

If the support comes, would it work like similarly to the following format? cdc_rsync E:\ F:\MyDrive\ -rv Yes, exactly.

I somewhat doubt, though, that cdc_rsync will perform any better than standard Windows tools like xcopy and robocopy in this case. Note that both tools can skip unchanged files by modification date. cdc_rsync only gives a benefit if the bandwidth between the local and the target system is lower than the disk IO speeds on the two systems. In case of a local copy the bandwidth is very high (basically, memory speed), so I don't expect any gain.

To illustrate what I mean, imagine there's a 1 GB file F:\MyDrive\foo.dat and the same file with a single byte inserted in the beginning at E:\foo.dat. In order to sync this file, cdc_rsync creates a temp file F:\MyDrive\foo.dat__cdc_rsync_temp__, copies a chunk of E:\foo.dat to it and appends the rest from F:\MyDrive\foo.dat. Finally, it replaces F:\MyDrive\foo.dat by F:\MyDrive\foo.dat__cdc_rsync_temp__. I highly doubt this is any faster than a plain copy.

I would suggest you try it out once the changes are in, and compare. I'm very curious what the results are.

[prarsena]

Thank you for your detailed response, and explaining that the benefit of cdc_rsync comes from the network improvements. I appreciate it. I will keep an eye on this project and test that out.

[rstupek]

When would a test version for windows->windows remote machine be available for testing? I would be willing to give it a try

[ljusten]

I hacked together a release build that contains the changes. I tested it to a local Docker container and it worked. Currently, I only have one Windows machine, so I can't test true remote copies, but I've applied for one.

https://github.com/google/cdc-file-transfer/suites/10439823585/artifacts/516488099

I'm super curious whether it works for you. Please let me know what issues you run into. Deploying the binaries properly is a surprisingly hard problem.

Note that we switched from scp to sftp, so you need to update your CDC_SCP_COMMAND to CDC_SFTP_COMMAND (but most likely you can just replace scp.exe by sftp.exe). Please see the readme contained in the zip.

[rstupek]

How do I get the copy to go on windows? Any hints on what command line to use would be appreciated

[ljusten]

Sorry for the delay, we had some distractions here lately. For local Windows syncing, i.e. copying files locally, you would just remove the user@host, e.g.

cdc_rsync C:\path\to\source\* C:\path\to\dest -rv

For remote Windows-to-Windows copies just add user@host:

cdc_rsync C:\path\to\source\* user@host:C:\path\to\dest -rv

The code will assume it's copying to Windows because the path starts with C: and deploy the right server file, i.e. cdc_rsync_server.exe.

[rstupek]

No worries on the delay, totally understand. I tried it and this is what I get: Error: CreateProcess() failed: The system cannot find the file specified.; Failed to start cdc_rsync_server process; Failed to start server

I have the remove drive mapped so security wise the source server is logged into the destination server.

[LEOS2021]

cdc-file-transfer-binaries-v1.0.0-x64 My code:

cdc_rsync D:\tmp\111\* D:\tmp\ggg -vr

Error: No remote host specified in destination 'D:\tmp\ggg'. Expected [user@]host:destination.

cdc-file-transfer-binaries-main-x64 My code:

cdc_rsync D:\tmp\111\* D:\tmp\ggg -vr

Port 51048: Server is listening EOF detected; Failed to receive packet of size 4; Failed to receive packet; Message pump thread is down; Failed to dequeue packet; Failed to receive SetOptionsRequest; Failed to receive options2113 file(s) and 1949 folder(s) found 2532 file(s) and 2323 folder(s) found

[danildavletov]

Error: No remote host specified in destination 'D:\tmp\ggg'.

I have the same issue. Any workarounds?

[graphixillusion]

Error: No remote host specified in destination 'D:\tmp\ggg'.

I have the same issue. Any workarounds?

Same for me. Same error using windows to windows local sync:

Port 51890: Server is listening
EOF detected; Failed to receive packet of size 4; Failed to receive packet; Message pump thread is down; Failed to dequeue packet; Failed to receive SetOptionsRequest; Failed to receive options582 file(s) and 21 folder(s) found