search

rust-lang / rustup

The Rust toolchain installer
https://rust-lang.github.io/rustup/
Apache License 2.0
6.13k stars 881 forks source link

delta compression for packages #653

Open llogiq opened 8 years ago

llogiq commented 8 years ago

Especially nightlies could benefit, both server&client-side.

spease commented 6 years ago

Stuck on a poor internet connection for the holiday, and I agree. xdelta3 is the best bindiff option that I know of.

lolbinarycat commented 2 weeks ago

perhaps RFC 3229 could be used for this.

lolbinarycat commented 1 week ago

did some testing with xdelta.

nightly-2024-06-10-x86_64-unknown-linux-gnu/bin/rustc is 2.6M
the 2024-09-19 nightly is also 2.6M

the delta between them is 525K (down to 487K when running with -9). that's about a 5x size reduction, pretty good!

keep in mind these are unideal conditions assuming no update in 3 months.

lolbinarycat commented 1 week ago

it's worth noting that xdelta3 is just a piece of software, vcdiff is the underlying file format.

unfortunatly RFC 3229 doesn't have much in terms of software support, but it's simple enough it could probably be implemented mostly with middleware shims.

djc commented 1 week ago

I think this is an interesting idea and think its time may have come. I think the implementation in rustup could be fairly straightforward, but I think the majority of work here will be on the backend. As such, recommend starting with an issue against https://github.com/rust-lang/infra-team (and please mention the new issue here if you do so that we can coordinate).