search

AppImageCommunity / AppImageUpdate

AppImageUpdate lets you update AppImages in a decentral way using information embedded in the AppImage itself.
https://appimage.org
MIT License
567 stars 57 forks source link

Investigate brotli, butler, and wharf from itch.io #109

Open probonopd opened 7 years ago

probonopd commented 7 years ago

Looks like they have a very elaborate and well-thought-out solution in place, what could it offer over zsync for AppImageUpdate?

https://github.com/itchio/itch/issues/683

azubieta commented 5 years ago

Can we close this ?

probonopd commented 5 years ago

Let's keep the "Investigate..." tickets that contain ideas around, unless we come to a conclusion that they are not feasible.

TheAssassin commented 5 years ago

Wrong project, though. Please open an issue in AppImageUpdate's tracker if you want to keep this information somewhere (or move it there).

antony-jr commented 5 years ago

@probonopd zsync and itch's solutions are completely different and zsync is a solution for what itch did , If I'm right , they use pre-calculated delta's , (i.e) they do delta's between two versions (like git does).

From zsync technical paper ,

Finally, there are the mechanisms traditionally used among programming projects — version control and diffs. The Linux kernel, for instance, is distributed by providing patches to get from one version to the next ([LKML FAQ]). For comparison with the other methods discussed, we can say that this method effectively pre-computes the changes between versions and then sends only the changes to the client. But it only works with a given fixed starting point. So to get from, say, 2.4.19 to 2.4.27, the user has to download the patch 2.4.19 -> 2.4.20, the patch 2.4.20 -> 2.4.21, and so on. This method is efficient if there are clear releases and the frequency of releases is smaller than the frequency with which users check for updates — it is less efficient when releases in the affected files are frequent, as there are then large numbers of patch files to manage and download (and these files contain enough data to construct not only the final file, but every intermediate revision).

As stated by the author , Developers who use AppImageUpdate don't really need to care about the starting point , which is a good thing , Imagine how hard would it be for the developer to save all old versions diff and patches in order to update from any old version to the latest one. So I think zsync is much suited for our use case. But we can do something about the I/O usage issue.

probonopd commented 5 years ago

f I'm right , they use pre-calculated delta's

If that is the case, then indeed that is not a solution for us.

antony-jr commented 5 years ago

@probonopd

From https://github.com/itchio/itch/issues/683#issuecomment-252418015

Our diff/patching system is very well suited to our needs, cross-platform, fast, etc. so there's really no need to fill here.

EDIT: Also the itch's docs on wharf which speaks about algorithm (diff/patching) ,

The diff algorithm is a modified version of the rsync algorithm, and has the following properties: Full access to the old version is not required — a series of hashes is enough.

What they mean above is that , the server(this also implies that we need special software on the server) does the hashing of the old version with respect to the block size and sends the hashes to client and the client simply uses these hashes to compare the difference by hashing the new version , This is really not required in our case since we don't care about upload bandwidth for now but still its a great idea if you wish to minimize the upload bandwidth.

And the rest is just patching using these precomputed diffs downloaded from the server.

But , If we reverse this process(that is swap the places of server and client) , it looks much more like an inefficient implementation of rsync. So I think there is no use in investigating this protocol.