systemd / casync

Content-Addressable Data Synchronization Tool
1.49k stars 117 forks source link

two bytes hex edited requires 55% remote chunks #252

Open felartu opened 3 years ago

felartu commented 3 years ago

Hi,

I'm testing casync to sync VM images, for now I'm using random binary data to test it, I have seen that when I change just 2 bytes on the binary file casync requires 55% chunks from remote server and only uses 45% seed data.

on webserver: 100MB.bin file created using dd from /dev/urandom

pwd

/root/test

ls -lah 100MB.bin

-rw-r--r--. 1 root root 100M May 31 21:10 100MB.bin

casync -v make 100MB.bin.caidx .

ls

100MB.bin 100MB.bin.caidx default.castr

mkdir -p /var/www/html/casync/

cp 100MB.bin.caidx /var/www/html/casync/

/bin/cp -rp default.castr /var/www/html/casync/

I now go to the client and download the file

On client:

casync -v extract http://remoteserver/casync/100MB.bin.caidx

Zero bytes written as sparse files: 0B Bytes cloned through reflinks: 0B Chunk requests fulfilled from local store: 0 (0%) Bytes used from local store: 0B (0%) Chunk requests fulfilled from local seed: 0 (0%) Bytes used from local seed: 0B (0%) Chunk requests fulfilled from remote store: 7539 (100%) Bytes used from remote store: 472.0M (100%) Time spent seeding: 1ms (0%) Total extract time: 19min 14.649s

I go to the server and open the binary file, change 2 random bytes and save.

hexedit 100MB.bin

casync -v make 100MB.bin.caidx .

cp -frp default.castr /var/www/html/casync/

/bin/cp -rp default.castr /var/www/html/casync/

On the client:

casync -v extract http://remoteserver/casync/100MB.bin.caidx --seed=/root/test

Zero bytes written as sparse files: 0B Bytes cloned through reflinks: 0B Chunk requests fulfilled from local store: 0 (0%) Bytes used from local store: 0B (0%) Chunk requests fulfilled from local seed: 3384 (44%) Bytes used from local seed: 213.2M (45%) Chunk requests fulfilled from remote store: 4155 (55%) Bytes used from remote store: 258.7M (54%) Time spent seeding: 29.660s (4%) Total extract time: 11min 29.752s

Any idea why it takes so many chunks for a small change?

Thanks,