nethappen / blocksync-fast

Fast block device sync with digest, designed to improve block-based backups.
https://nethappen.github.io/blocksync-fast/
Apache License 2.0
10 stars 1 forks source link

Hi, Can we add support stdin or pass fd as src_device? #2

Open xinminglai opened 1 month ago

xinminglai commented 1 month ago

Now, I tried with something like,

xzcat /path/to/image.xz | blocksync-fast -s - -d <dst_device> doesn't work.

blocksync-fast -s <(xzcat /path/to/image.xz) -d <dst_device> doesn't work either.

The reason this feature will be useful is, We can have compressed image archives, with unused spaces to be zero, The source image will be very tight. Without stdin support, We have to use a large tmp folder with a more time consuming step.

Thanks.

nethappen commented 1 month ago

Hi, thank you for your interest in the blocksync-fast program. After reviewing the issue, I have already implemented functionality to handle input from STDIN as the source image. You will only need to provide an additional parameter for data size, as that was the only problem. I am currently testing the program and making a few minor adjustments. The new version should be available within 24 hours. :white_check_mark: :smiley:

xinminglai commented 1 month ago

Hi, I haven't tried yet, As busy with other things.

I just saw your reply, So an enhancement, Is it possible to remove the additional data size requirement? The reason is, when we use dd command, We don't need to provide the size of if= and of= parameter. dd's logic is,whenever in-file or out-file reaches EOF, dd quits. if in-file is smaller than out-file, the command ends successfully. if in-file is larger than the out-file, there will be an warning to show in-file EOF ealier than out-file. What I think is, blocksync-fast gets the same behavior as dd. So people can think this as an smart dd command in some use cases.

Thanks.

nethappen commented 2 weeks ago

Yes, I understand this request. The problem is that the program needs to know the size of the synchronized data in advance, e.g. for writing the header in the Digest file, if it uses such functionality. The functionality requires some rethinking and rewriting of some program functions. I will try to include this functionality in future versions.