Understand how to leverage zsync special handling of compressed files

[probonopd]

Need to understand what http://zsync.moria.org.uk/ has implemented regarding

Handling for compressed files

rsync is ineffective on compressed files, unless they are compressed with a patched version of gzip. zsync has special handling for gzipped files, which enables update transfers of files which are distributed in compressed form.

The special handling of compressed files is, as far as I know, entirely new and unique to zsync. The combination of client-side rsync and HTTP is also unique to zsync, to the best of my knowledge.

Also need to understand if we can leverage the same for

• Building the blocks for zsync2 but targeting our squashfs files plus the compression we are using there, and (informed by this understanding) potentially optimize the compression and block size used in our squashfs files
• Draw conclusions for the chunking of files for peer-to-peer distributions (ideally: using the same blocks/chunking mechanism)

Action should only be taken after a discussion and ideally getting input from @whyrusleeping, too.

[TheAssassin]

What exactly is the purpose of this issue?

[probonopd]

To eventually implement an efficient chunking mechanism for libzsync2 and possibly ipfs which will make delta binary downloads and possibly p2p downloads smaller.

[shoogle]

See also https://github.com/systemd/casync/. From Wikipedia:

casync (content-addressable synchronisation) is a Linux software utility designed to distribute frequently-updated file system images over the Internet.