Case-Study of top `rgb` dependent crates

[ripytide]

After discussion of 0.9 and the improvements that may take breaking changes I thought it might be a good idea to study some of the largest crates dependent on rgb and how easily they'd be able to migrate to a version 0.9. I'll list the crates in order of downloads from this crates.io page of rgb's dependents.

crate	upgrade without semver	description
image	✅	optional dependency only used internally for the avif decoder
inferno	❌	re-exports RGB8
ansi_colours	✅	optionally implements one of their public traits some rgb types
gif-dispose	❌	re-exports RGBA and RGB
resvg	✅	only used internally
csscolorparser	✅	optionally implements From to their internal Color struct from two rgb types`
lodepng	❌	re-exports lots of RGB types
winconsole	❌	optionally re-exports ConsoleState which contains a field of type [RGB8; 16]
opencv	✅	optionally implements one of their public traits on some rgb types
resize	❌	re-exports Gray, RGB and RGBA
fast_image_resize	✅	only used internally
ravif	❌	re-exports RGB8 and RGBA8
svgfilters	❌	re-exports RGB8 and RGBA8
femtovg	❌	re-exports then entire rgb crate
mozjpeg	?	uses the rgb::Pod trait as a bound in two functions
textplots	❌	takes RGB8 as an parameter to a trait method
imagequant	❌	re-exports RGBA
oxipng	❌	re-exports RGB16 and RGBA8
smart-leds-trait	❌	re-exports RGB8 and RGB16 and pub type RGBW = RGBA eww
color-thief	❌	re-exports RGB
dssim-core	❌	re-exports RGB and RGBA and many functions take &[RGB<u8>] as a parameter
gifski	❌	re-exports RGB8 and RGBA8

I think it's worth noting that the crates here only include libraries and not applications so I wouldn't rely on these results as an fair representation of rgb's usage across the rust ecosystem.

[ripytide]

I think I'm still in favor of creating a 0.9 breaking release since all the crates who are depending on 0.8 don't have to upgrade if they don't want to make a breaking release, their users can still use their re-exports of the 0.8 version. New applications or libraries who don't mind making a breaking change can move to 0.9 if they want to, this way everyone gets the best of both worlds.

The only disadvantage to this is that users who want to integrate crates using different versions 0.8 vs 0.9 might have to throw in a conversion from one to the other. This conversion shouldn't be too hard in the simple case of RGB0.8 <-> Rgb0.9 but might be trickier for conversions like Vec<RGB0.8> <-> Vec<RGB0.9>?

[kornelski]

Thanks for the study. However, I still think breaking the compatibility completely by replacing the structs is not appropriate for this crate. The existing structs are not that terrible to warrant a harder upgrade path. Gray.0 is slightly ugly, but it can have a getter (px.luma()) or it can be used as let Gray(gray) = px.

The main purpose of this crate is the interoperability. RGB(A) structs are not rocket science, and anyone can define and export their own. The main thing this crate has going for it is this interoperability, and a completely new 0.9 would reset it to zero. Re-exports help when using one crate at a time, but not for combining multiple crates, which is the main purpose of this crate.

Small breakage that requires updating interfaces/traits to new ones is tolerable, especially that it may be doable gradually by adding #[deprecated] and forward-compatibility shims to 0.8, but an all-or-nothing upgrade that makes 0.8 and 0.9 coexistence difficult is too much. The Rust ecosystem has laggards that rarely or never upgrade. lib.rs estimates that half of crates using rgb are not actively maintained (active users/direct rev deps). It's the worse half, but it may include some crates that would prevent others from upgrading. I've seen my crates have download spikes unexplained by their reverse deps on crates.io, so the rgb has closed-source users too.

Rust's error about type mismatch between same type in two crate versions is annoying, because it doesn't know crate versions, and there's 50% chance that it may present incompatibility "backwards" suggesting conversion to 0.8. Rust doesn't have safe transmute yet, so that will require custom conversion glue code to be added manually, and then removed, if the other dependencies catch up with upgrades.

[ripytide]

Fair enough, if interop is the priority then not making breaking changes is understandable.

I might be too idealistic/naive/optimistic but I'd like to believe that the rust ecosystem is well-maintained enough to manage breaking updates to popular crates, since the alternative would be that it isn't and we could one day reach a plateau where no-one can make improvements since they are too scared of causing incompatibilities with existing crates.

I think the only solution might be for me to create a new crate with this alternate goal, if only to satisfy my idealism.

I will close the non-semver compatible changes from my tracking issue and leave it open should anyone else want to continue the work on a semver-trickable v0.9 for this crate.

[kornelski]

The crucial difference is dynamics created by inter-dependencies between crates.

If two versions are interoperable, then you can upgrade your crate to 0.9 immediately, without waiting for your dependencies, and also your crate immediately becomes a 0.9-compatible user for others. The entire ecosystem can start using 0.9 immediately, and nobody has a reason to use 0.8 any more. Unmaintained crates will still lag, but won't be holding others back.

OTOH if two versions don't easily interoperate, then you can't easily upgrade until your deps upgrade, and the deps can't upgrade until their deps upgrade. It creates a chicken-egg problem. Other crates may hesitate to upgrade even if technically nothing stops them, because it cuts them off from all their existing 0.8 users. Even new crates may decide to stick with the more popular version, until the new version slowly builds a majority.

There's also psychological effect in amount of work required. If it's possible to upgrade with little code changes, users can upgrade immediately, and become committed to the new version. It's not a big problem if they put off fixing deprecation errors and interface changes for later. OTOH if an upgrade changes too much at once, and makes build look severely broken, users may put off the upgrade for later.

semver-major upgrades in the Rust ecosystem are not easy:

• https://lib.rs/crates/hyper/rev
• https://lib.rs/crates/rand/rev
• https://lib.rs/crates/env_logger/rev
• https://lib.rs/crates/remove_dir_all/rev
• https://lib.rs/crates/cargo_toml/rev
• https://lib.rs/crates/serde_qs/rev
• https://lib.rs/crates/base64/rev

So if you have a vision for a new interface, I think it's easier to do it "boil the frog" style by adding new interface as optional first, and slowly removing the old one, rather than making all users take the plunge all at once.

[ripytide]

OTOH if two versions don't easily interoperate, then you can't easily upgrade until your deps upgrade, and the deps can't upgrade until their deps upgrade. It creates a chicken-egg problem.

Not really, the chicken and egg problem is a cyclic dependency, but rust dependencies are acyclic, there will always be a crate at the bottom that doesn't rely on other crates using rgb and so can upgrade to 0.9 which will then start a wave of upgrades up the dependency chain until every crate is on 0.9. The interop between 0.8 to 0.9 is only required while this upgrade wave is in flight.

Interop isn't that impossible, the embedded-hal crate did this and the whole ecosystem just took it on the chin supporting multiple versions until we finally got to 1.0.

So if you have a vision for a new interface, I think it's easier to do it "boil the frog" style by adding new interface as optional first, and slowly removing the old one, rather than making all users take the plunge all at once.

But the perfect "finished" form of this library may take backwards-incompatible changes of which we have discussed several by now. This cannot be done gently as far as I am aware otherwise I would have agreed to do it that way.

[kornelski]

https://lib.rs/crates/embedded-hal/rev

[ripytide]

I'm not sure what you are implying with all these links to reverse dependencies? Practically all of embedded-hal's reverse dependencies support v1.0 and v0.2. The lib.rs page doesn't seem to support showing the dependent version(s) of crates which depend on multiple versions of the same crate which is a bit misleading in this case.

[kornelski]

The itemized list of crates is necessarily simplified to show only one version, but the chart above it is accurate for all versions. It does fully recursive deep dependency resolution for all crates in the index. You can cross-check it with downloads chart on crates.io that shows 1.0 is about 1/3rd of downloads.