search

AmesCornish / buttersink

Buttersink is like rsync for btrfs snapshots
GNU General Public License v3.0
193 stars 18 forks source link

Add optional compression and resume support to remote (with ssh) btrfs send/receive #34

Open Fantu opened 7 years ago

Fantu commented 7 years ago

Hi, thanks for this good software, it is very useful. I found some possible improvements to make it optimal also with big remote transfert with very low bandwidth connections and possibile errors. I think can be very useful add for remote case optional compression send with resume and bandwidth limit support. This seems possible saving btrfs send output to file, optionally compress it, send it with rsync that can resume partial files and have also bandwidth limit support (that can be also added optional), on remote destination uncompress it (if needed) and do btrfs receive of the file. If fails maintain the "btrfs send" file and resume the transfert (with rsync). I did some fast manual tests to check if works and how: I did btrfs send with -f for save to file instead I compress it to gzip that seems a good compromise between space and used resources/time. After I send it with rsync --partial for resume support (including simulate common problem), for example I did: rsync --partial --progress --rsh=ssh fullsend.gz remotehost:/mnt/btests/fullsend.gz Is possible add also bandwidth limit adding --bwlimit= After I did uncompress and btrfs receive on destination. All is ok except the uuid of snapshot for use it for diffs with buttersink, I not found a btrfs-tools command to change uuid same as buttersink done but I suppose that including this improvements in buttersink is possibile maintaing actual buttersink features.

I don't have experience with python, can someone with experience add these features that I think are very useful (or essentials in many cases)? Optimal result I suppose can be these additional parameters: -r --resume-support save the btrfs send/receive data to file and make possible transfert it with possibility of resume, useful for remote transfert avoiding to redo all from start in case of error or needed "transfert pause" -c --compress compress the btrfs send/receive data with gzip, useful for low bandwidth, require the -r -b --bandwidth-limit limit the remote transfert, require the -r

Thanks for any reply and sorry for my bad english.

AmesCornish commented 7 years ago

Fantu, thanks for the suggestions. I can see how it would be useful to have resume and bandwidth limiting. I don't have time to work on this myself right now, but I'll keep it on the list and hopefully I or someone else will tackle it in the future.