Dependencies resolution with `--minimal-versions`

[matklad]

Implementation PR: https://github.com/rust-lang/cargo/pull/5200 Docs: https://doc.rust-lang.org/nightly/cargo/reference/unstable.html#minimal-versions Issues: https://github.com/rust-lang/cargo/labels/Z-minimal-versions

Steps:

• Document best-practices (https://github.com/rust-lang/cargo/issues/5656)

Unresolved questions:

• do we want to "impose" this feature on the ecosystem? Currently, everything seems to work fine due to eager dependency resolution. Adding --minimal-versions has costs: one-time ecosystem transition cost, cost to run CI job for minimal versions, cost to actually update minimal versions. There's anecdotal evidence that wrong minimal versions actually are a problem: https://www.reddit.com/r/rust/comments/8ob598/rust_minimum_versions_semver_is_a_lie/e027mtz/.

• should we implement "--minimal-versions-for-me-but-not-my-dependencies" as well, to make the initial roll-out of this feature easier?

Stabilization TODO:

• change -Z minimal-versions to just --minimal-versions and add it alongside --frozen/locked,
• this will require community-wide effort to make the things work, so a special announcement should be prepared. Announcement should describe the links problem and solution.

[Eh2406]

I would like to cc some persons:

• @klausi you did the implementation and many of the prs to get CI using it, including hear on cargo.
• @illicitonion you wrote up a great blog post on kicking the tyres on this feature.
• @matklad for your, already linked, articulit descriptions of the downsides of "imposeing" this on the ecosystem.
• @alexcrichton as you have encouraged some care when it comes to rolling out this feature.
• @dtolnay as you have seen real world bugs caused by not having this.
• Who did I miss that holds strong views for/against this?

I'd like to see us come to a shared understanding of what we would recommend users do with this feature both at rollout and at steady state in the idealized future.

• A comically-pro-this-feature opinion thinks a crate is only valid if it can be built with minimal-versions, so we should check at publish time with no override, and we should yank all existing crates that don't meet this standard.
• A comically-against-this-feature opinion thinks that the meaning of Cargo.toml is what cargo dose with it, so if ">=0.0" as a version requirement gets cargo build to work than what is the harm, So we should not stabilize this feature.

I think we are all in the middle, and probably closer than we think, we just need to articulate it. To start it off I will propose a position as a strawman, so you can point out where I am wrong.

All crates should try minimal-versions. If it can be made to work it should be added as a separate test in CI. As then things that depend on your crate will be able to use minimal-versions easily. yanking is generally not called for, only to be used when it predates rust 1.0.0 and it is a real problem for the ecosystem. If CI resources are tight, then minimal-versions can be combined with one of the other runs, like the minimal rust version or the beta run.

[klausi]

I'm currently not that interested in how strongly we should impose this option on the ecosystem. First we should make the minimal-versions option work properly. Not even cargo itself can be compiled with minimal-versions as explored in rust-lang/cargo#5275.

Repeating the open points here:

1. cargo publish always needs to send the links key to the registry. This needs to be stabilized as default behaviour in stable cargo.
2. Mass update the crates.io registry to add the links key where it is missing right now.
3. Stabilize the -Z minimal-versions flag into --minimal-versions on Cargo stable

[Eh2406]

Well one small peace is done. At this time cargo publish from stable pushes the links key.

[klausi]

Oh cool, that is good news!

The inconsistent crates.io registry is still a problem. @alexcrichton said in rust-lang/crates.io#7310 that getting minimal-versions to work smoothly was not a priority then, so he didn't want to mass update the creates.io index. Is that still the assumption?

@SimonSapin argued that people will always use old cargo publish versions and by that make the crates.io registry inconsistent again. In order to prevent that we would have to implement server side rules on crates.io to prevent entries without links attributes. Which means breaking those old cargo publish versions.

For experimenting I'm maintaining a crates.io registry fork, that I update very irregularly. It has the links attribute fixed for some popular crates such as curl-sys.

CARGO_REGISTRY_INDEX=https://github.com/klausi/crates.io-index cargo build -Z minimal-versions

[Eh2406]

Is that still the assumption?

That is one of the questions I think we need to address. :-) Hopefully, @alexcrichton and @matklad will have a chance soon to articulate what there current thoughts are.

What we decided to recommend needs to work and fairly smoothly. That means that if there are hard changes that need to be made to make it work then we need to be willing to do them. Correspondingly if we are not willing to make the big changes then we need to recommend something smaller.

Big changes may include some or all of:

• foundational and otherwize stable crates releasing new versions just to bump minimal dep.
• yanking versions too old to be worth making a new release for.
• manually updating the index to add the links attribute, and requiring it to be correct for all future uploads.
• other ideas?

Recommend something smaller may include some or all of:

• recommending a --minimal-versions-for-me-but-not-my-grand-dependencies
• endorsing a community maintained fork of the index that simulates the "Big changes"
• making it clear that this is an optional "nice to have" and that it is not worth doing unless your dependencies are already doing it.
• having some kind of log of things cargo discover do not work, that gets included into the index if they get tried again. (this would require a lot of design work.)
• other ideas?

[alexcrichton]

I think we'll definitely want to get this working with the main index, and I think we probably just want to keep going as-is, fixing up crates and publishing them as we discover mismatches. In that sense I think it might be good to do some more work to get some of the "base crates" working and then perhaps make a post on internals asking for testers so we can discover new crates and publish new versions

[illicitonion]

I got a non-trivial crate (https://github.com/pantsbuild/pants/tree/c8b42cb52eca9acbca98eaaf9599a47bc7b1f51e/src/rust/engine) compiling with -Z minimal-versions, and have started sending out a few PRs to the ecosystem.

My two big questions from this process are:

1. What version of the log crate should libraries depend on? Nothing before 0.3.4 builds with post-1.0 rust. protoc depends on 0.* (https://github.com/stepancheg/rust-protobuf/blob/264debff3cd1cb26048925e90bec998941c2a328/protoc/Cargo.toml#L17), and many crates depend on 0.3. Most things which depend on 0.3 work with 0.4 It would be great to have some published advice on this one, as it's a slightly weird dependency in the first place. log itself notes some compatibility guidelines (https://docs.rs/log/0.4.2/log/#version-compatibility), but it would be nice to expand that to a firm suggestion ("You should always depend on the most recent version where possible" or "You should always depend on the oldest version which works for your library", or something else)
2. If we decide that --minimal-versions is something people should strive to support, how should people handle dependencies which aren't actively maintained, and which need updating? If a transitive dependency is the only problem with an otherwise functioning dependency which hasn't been touched in 2-3 years, and which doesn't respond to PRs, what should people do? Fork the crate? Try to get the bad transitive dep yanked? Give up?

[alexcrichton]

@illicitonion nice!

Those are indeed good questions too :). You can somewhat force the process by having some crate have a higher version bound (aka requiring 0.3.10 of log synthetically) but that's not a great solution to either problem. I think for now the best advice we'd have is "try to send a PR to the crate and get a new version published" but that indeed reduces the usability of this feature :(

[Eh2406]

Actually I quite like the suggestion,

If you minimal-versions build is broken by log then add a dependency on 'log = "3.10"' to your Cargo.toml. If you can't do to your crate already depending on 'log = "4"' then depend on the helper crate 'logs-that-works-with-minimal-versions = "1"' witch is just a Cargo.toml that has 'log = "3.10"'.

I don't think "try to send a PR to the crate and get a new version published" reduces the usability, I think it is begging the question "If a transitive dependency is the only problem with an otherwise functioning dependency, and which doesn't respond to PRs, what should people do?".

[alexcrichton]

A snag I've now thought of as well: from time to time crates will break due to language/compiler changes, but we're generally pretty good about ensuring that the most recent version on crates.io always builds and point releases are updated. This means, however, that lots of crates' CI will break when that Rust version is published, because not everyone will say they require the newer version of the crate.

[matklad]

@alexcrichton I think that depends on the CI setup. It seems to me that long-term the CI job with --minimal-versions should also use the minimal supported Rust version(MSRV): this is needed so that crates can bump minimal supported rust version in minor release, and it also saves one CI job as well.

However, the day when your MSRV supports --minimal-versions is far away; in the meantime, the following guideline should work:

• if your MSRV has --minimal-version, add a single CI job which both sets MSRV and --minimal-versions
• otherwise, add two jobs:
  • MSRV with a lockfile (otherwise CI breaks when your deps bump their MSRV)
  • --minimal-version with rust 1.xy, where 1.xy is the first rust release to support --minimal-versions

[Eh2406]

@matklad in the meantime can it be done in one CI job now with cargo +nightly generate-lockfile --minimal-versions && cargo +MSRV test?

[matklad]

Excellent idea @Eh2406!

[alexcrichton]

@matklad makes sense to me! And I like @Eh2406's idea as well, that should make an excellent suggestion for how to best use this flag

[Eh2406]

This is a list of foundational crates with versions that do not build on modern rust. Purging these from tomls is the "startup cost" of getting minimal-versions working. This is in a format that can be copied into a tomls to fix each dep.

• winapi = "0.2.7"
• libc = "0.1.x" dont know x yet
• log = "0.3.4"
• num = "0.1" update to num-traits

I will keep this up to date as I find more, and one day make a working-with-minimal-versions-hack crate with this as it's starting toml.

[Eh2406]

So @dwijnand asked me to write up how do figure out what to do when a minimal-versions CI job fails.

So hear gose.

1. Reproduce locally. This involves checking out the branch and running what the CI job does. IntelliJ and RLS has a habit of updating a lock (without -Z minimal-versions) file on Cargo.toml change, So just to be safe I tend to close my editer when investigating.
2. Identify the problematic dependency. This is often easy to do by reading the path where the errors occurred. For example this hit an error in C:\Users\appveyor\.cargo\registry\src\github.com-1ecc6299db9ec823\libc-0.1.1\rust/src/liblibc/lib.rs:79:46 So that is libc-0.1.1 It is significantly harder if it is a build script that errored, as it can be any build-dep of that dep.
3. Determine how that ended up in your tree. I find cargo tree extreamly helpful for this. So I do cargo +nightly generate-lockfile -Z minimal-versions && cargo tree -p libc:0.1.1 -i Cargo tree also updates the lock (without -Z minimal-versions) if it is not fully up-to-date, so I always run the two commands in one line. Also the format for specifying a dep uses a : instead of a - so watch out for that.
4. Find something to add to your toml to fix the dep. Best case is that the most recent version of the thing you actually depend on builds with minimal-versions requiring to that can solve a large swath of problems. For example curl = "0.4.13+" . Next best is that newer versions the thing you actually depend on no longer require the thing that causes the problem. This can solve the immediate problem, but can release new problems. For example git2 = "0.7.3" Next is to add a synthetic deps. I find that opening a tab for each compatible version of a deb on crates.io makes it fairly easy to binary search for when a dep got bumped.

[ehuss]

As discussed in rust-lang/cargo#6636, dealing with breakage is difficult and can take a large amount of time to diagnose and fix, with questionable benefits. Generally minimal-versions won't be useful unless everyone is using it (which IMHO is unlikely).

@Eh2406 brought up in the team meeting today an alternate implementation that only enforced minimal versions on direct dependencies. (Essentially forcing = requirements.) To me, this sounds like it would be much easier to use. It sounds like something that might be worth experimenting with.

[ghost]

We inadvertently broke some crates by publishing a Crossbeam release without checking that it builds with minimal dependencies: https://github.com/crossbeam-rs/crossbeam/issues/312

I'd love to have a minimal-versions check in our CI, but it's impossible to build Crossbeam because the whole crates ecosystem is broken, unfortunately. There are two things we should do, IMO:

1. Add an option like --minimal-versions-for-me-but-not-my-dependencies as @matklad suggested. That would allow us to at least have some checks in CI rather than nothing.

2. Add an automatic check on cargo publish. We can issue a warning if the crate doesn't build with minimal dependencies, and maybe later in the future starting issuing errors.

[dekellum]

Somewhat related to the process outlined by @Eh2406 above...

As a prerelease exercise, I maintain and check in a Cargo.lock on a dedicated minimal-versions branch. The maintenance of that lock file isn't pretty, but I'm getting faster at it every time I do it, and I've avoided a few bugs with my own minimum versions thus far. What seems to work best for me is keeping a companion script with a bunch off cargo update -p <package> --precise <downgrade-version> lines: dekellum/body-image@c16efc33 Note this includes minimal direct dependencies as well as some interesting, working, minimal-ish transitive dependencies.

Then I not-so-Continuously Integration test that branch.

Said another way: if you are lucky enough to produce a minimal-version Cargo.lock file that actually builds, definitely check-in somewhere in VCS, so you can continue to use it or compare it with future attempts!

[Eh2406]

Testing --minimal-versions for Cargo on windows was "temporarily" removed in rust-lang/cargo#6748 do to the way rand is broken.

[ehuss]

I was reading the public private dependencies RFC and I noticed it mentioned this feature:

cargo publish will resolve dependencies to the lowest possible versions in order to check that the minimal version specified in Cargo.toml is correct.

Just leaving a note here about this because I wasn't aware about that until now, and pub/priv deps might make this issue more relevant.

I'm a little uncertain about issuing warnings if cargo publish fails. If a build fails, it would have to build twice, and for some crates it may take a very long time to build twice. If cargo only did minimal versions for direct dependencies, I would be comfortable with just building once with minimal deps and make it a hard error if it fails.

[BurntSushi]

I had to remove the minimal version check from regex's CI tests because of rand. The rand crate continues to, for example, advertise support for libc 0.2.0 even though rand cannot work with that version of libc. (There may be other incorrect dependency specifications in rand, I'm not sure.)

We aren't going to get anywhere if core crates refuse to maintain and test correct dependency specifications. I don't know how to convince them to do it either. In the case of rand, it looks like CI resources are to blame? But I'm not sure.

[ehuss]

@BurntSushi yea, I think this feature is dead as-is for now. Someone needs to implement minimal-versions-for-me-but-not-my-dependencies and see how it goes from there. I'm uncertain how difficult that will be, I haven't looked at it myself.

[vks]

@BurntSushi FWIW, the incorrect libc version in Rand was fixed. It's true that the Rand CI is not set up to test for the minimal versions, but I think pull requests correcting incorrectly specified minimal versions are accepted.

[BurntSushi]

@vks Sorry, but I'm not going act as rand's unofficial CI by submitting PRs whenever downstream detects that rand's minimal versions aren't correct. I've either just been removing the minimal version check on my end, or removing rand as a dependency where possible.

[dekellum]

FWIW: In my active projects, including a public crate with Cargo.lock below, I've avoided updates to rand for ~5 months now, while taking other updates, due to induced duplicates (rand-core 0.3.1 trick-shim + 0.4.0) and its perceived stability threat.

https://github.com/dekellum/body-image/commits/dev/Cargo.lock

Meanwhile, I've infrequently but successfully continued, semi-manual minimal version testing:

https://github.com/dekellum/body-image/compare/dev...minimal-versions-8

[josephlr]

We just ran into an interesting bug (or feature, not sure) around -Z minimal-versions.

getrandom (the OS-specific code for rand) attempts to have correctly specified minimum versions by running a check in our CI:

cargo generate-lockfile -Z minimal-versions
cargo test

However, even when this passes, we still get bugs like https://github.com/rust-random/getrandom/pull/112. The basic problem can be demonstrated in the following example:

• Crate A v0.2 unconditionally depends on crate B v0.2.2
• Crate A v0.2 has an optional dependency on crate C v0.1
  • Could also be a target-specific dependency, or dev dependency
• Crate C v0.1 then depends on B v0.2.6.

Now let's say Crate A starts using features added in B v0.2.5. Running cargo generate-lockfile -Z minimal-versions will select B v0.2.6 as the "minimal" version of Crate B, so the above checks will pass, despite Crate A now having an incorrect version specification.

Does anyone know of a way to fix this? Is it possible to build the minimal version dependency tree for just the mandatory dependencies, or with a specific set of Cargo features?

EDIT: -Z avoid-dev-deps doesn't seem to help the situation. It's also not clear what that flag even should do.

[dekellum]

It would seem that cargo -Z minimal-versions generate-lockfile should be aware of, and lock results conditional upon, what features are selected either by default, or via additional feature arguments to that command. You also demonstrate that the target can effect default features. But as of nightly-2019-09-25, the generate-lockfile command does not accept feature nor target related flags.

Beyond the getrandom case but I suspect related: how does -Z minimal-versions generate-lockfile interact with workspaces, e.g. if it is workspace and CWD package aware, and can yield different results if run from different package directories within a workspace tree? Consider the case where two workspace packages had different minimums for the same dependency.

[mathstuf]

There might need to be additional cargo syntax here (or I'm missing how this can be done today).

• crate A has a version spec on its dependency X of >= 0.0.x, < 0.y
• crate A exposes X data types in its API, but the name of the crate is not necessarily visible
• crate B depends on version A (any version, doesn't matter)
• crate B uses the exposed X data type

However, B uses X APIs that A does not need (so A isn't going to bump its minimum version), but B needs to bump the minimum version A needs to expose for it to work. Adding a direct dependency on X is liable to get caught up in cargo-udeps checks since the crate is not directly used.

Is there some way for B to bump the minimum version of X it needs from A somehow without depending directly on X (without direct usage)?

The realworld case I hit this with is yaml-rust and serde_yaml (linked-hash-map is X here).

I don't think this should block, but defer to maintainers about how bad the problem is. This seems like a hairy problem and adding Cargo.toml stuff hasn't seemed to be the easiest thing to do.

[mathstuf]

Digging in more, serde_yaml does require a new enough linked-hash-map itself, but it still ends up with the really old API from yaml-rust's exposure. I guess a PR to yaml-rust might fix this specific problem, but is that problem something to worry about overall?

[mathstuf]

Also seeing this with serde and serde_derive when using the derive feature for serde itself. serde just requires serde_derive = "1.0" which ends up at 1.0.0 and doesn't know about many of the attributes used. I'll stop posting more examples here (though I'll write down instances of this I find in the meantime if that data ends up being useful), but that's a more serious one than the yaml crate at least.

[kornelski]

I think this feature is salvageable. It's a matter of yanking a few popular crates that have too lax dependencies.

I've started a crusade against "*" deps:

[Nemo157]

Someone needs to implement minimal-versions-for-me-but-not-my-dependencies and see how it goes from there.

I finally got round to figuring out how to do this on my project:

cargo update $(cargo metadata --all-features --format-version 1 | jq -r '. as $root | .resolve.nodes[] | select(.id == $root.resolve.root) | .deps[].pkg | . as $dep | $root.packages[] | select(.id == $dep) | "-p", "\(.name):\(.version)"') -Z minimal-versions

[Arnavion]

@Nemo157 I don't think that command is completely correct. With a crate that has a single dependency chrono = "0.4", the cargo metadata command emits -p chrono:0.4.13. Running cargo update -p chrono:0.4.13 -Z minimal-versions generates a lockfile with chrono 0.4.0 and num 0.1.0 (chrono 0.4.0 depends on num = { version = "0.1" }) instead of chrono 0.4.0 and num 0.1.42

It seems to me what it needs to do is start with a non--Z minimal-versions lockfile, then run cargo update -p chrono --precise 0.4.0, but I can't see if cargo metadata has a way to give us chrono 0.4.0

Edit: This might work:

1. Start with a -Z minimal-versions lockfile.
2. Run cargo metadata and record the selected version of every direct dep (cargo metadata --no-deps | jq '.packages[].dependencies[]').
3. Generate a new non--Z minimal-versions lockfile.
4. Run cargo update -p --precise for every dep from step 2.

[Nemo157]

Ah yes, I forgot to mention that you should start with a normal maximal versions lockfile. It also will not correctly update newly added dependencies-of-dependencies to their maximal version, but that would require an iterative solution to perform externally to Cargo, and doesn’t come up in practice much.

[Arnavion]

Starting with a normal maximal versions lockfile doesn't change what I wrote.

$ cargo update $(cargo metadata --all-features --format-version 1 | jq -r '. as $root | .resolve.nodes[] | select(.id == $root.resolve.root) | .deps[].pkg | . as $dep | $root.packages[] | select(.id == $dep) | "-p", "\(.name):\(.version)"') -Z minimal-versions

    Updating crates.io index
    Removing autocfg v1.0.0
    Updating chrono v0.4.13 -> v0.4.0
      Adding num v0.1.0
    Removing num-integer v0.1.43
    Removing num-traits v0.2.12

The point is -Z minimal-versions applies to all crates added in that cargo update command, not just the one specified by -p. Since the 0.4.13 -> 0.4.0 transition requires adding a new num 0.1 dependency, this means -Z minimal-versions also applies to it.

[Arnavion]

Here's a version that implements the algorithm I specified in my first comment's edit:

cargo-update-minimal() {
    rm -f Cargo.lock
    cargo update -Z minimal-versions
    dep_names="$(cargo metadata --all-features --no-deps --format-version 1 | jq '.packages[].dependencies[] | "\(.name)"' -r)"
    declare -A dep_versions
    for dep_name in $dep_names; do
        dep_versions[$dep_name]="$(cargo metadata --all-features --format-version 1 | jq ".packages[] | select(.name == \"$dep_name\") | .version" -r | tail -n1)"
    done

    rm Cargo.lock
    cargo update

    done=0
    until [ "$done" -eq '1' ]; do
        done=1
        for dep_name in "${!dep_versions[@]}"; do
            cargo update \
                -p "$dep_name:$(cargo metadata --all-features --format-version 1 | jq ".packages[] | select(.name == \"$dep_name\") | .version" -r | tail -n1)" \
                --precise "${dep_versions[$dep_name]}" || done=0
        done
    done |& grep -v 'Updating crates\.io index'
}

Output:

    Updating crates.io index
    Updating crates.io index
    Updating crates.io index
    Updating chrono v0.4.13 -> v0.4.0
      Adding num v0.1.42
      Adding num-iter v0.1.41

The lockfile has the desired outcome of chrono 0.4.0 and num 0.1.42

Notes:

1. If you have a dependency on both a package and its direct dependency, the script may get stuck in a loop being unable to downgrade the second dependency. Say you have a dependency on both native-tls and openssl, then if your openssl dep is too lax the script may get stuck in a loop trying to downgrade openssl to below what the already minimal native-tls allows. In this case abort the script and change your direct dependency on openssl to match the minimum that native-tls requires and rerun.

2. If you have a dep on serde and find issues with serde_derive being downgraded to a different version than serde, ie either your code doesn't compile or your deps don't compile because of the mismatch, then update serde to ^1.0.103. 1.0.103 is the first version that started enforcing serde_derive == the same version as the serde crate.

---

Edit: Fixed quoting to work when there is more than one direct dep, and removed the need for evaling a string as a command.

Edit 2: Added --all-features to support optional deps like Nemo157's original script.

Edit 3: Run cargo update --precise in a loop in case updating a dep fails the first time because a later dep must be downgraded first. For example, this happens with proc-macro2 = "1" and syn = "1" because syn must be downgraded before proc-macro2 can.

Edit 4: Handle the case where the lockfile ends up with multiple versions of a single dep, one of which is direct and the others are transitive. Pick only the latest version on the assumption that the earlier versions are transitive deps, and use the latest version as the predicate so that the earlier ones aren't affected. Also, filter out the Updating crates.io index lines from the loop to reduce noise.

[untitaker]

@xfix has recently opened issues across multiple crates to yank versions that do not compile on latest Rust. Others before him have asked crates to be yanked by other criteria.

While it achieves the desired effect, I think this is an insufficient way to drive forward development and feasibility of --minimal-versions, as it leaves the entire ecossystem in a broken state for users of older Rust versions: older versions are yanked, and newer versions are unlikely to compile due to requiring new language features. I would urge y'all to reconsider this strategy.

[kornelski]

@untitaker There's nothing to worry about yanking of old broken crates. It helps users of both new and old Rust versions.

First of all, the crates that need to be yanked are mainly from <=2015, and don't work with Rust 1.x at all. They're not crates for what we call Rust, but crates for a previous experimental unstable language that predated Rust 1.0.0 release.

Even if we needed to yank some non-prehistoric crates that actually work with Rust 1.x, then their users wouldn't be affected much, because yanked crates can still be downloaded and used by users who have a lockfile.

What about users of very old Rust who don't have a lockfile? They need the minimal-versions feature! It's almost impossible to compile any Rust crate with old Rust versions and the latest-version dependency resolution. It's an endless whack-a-mole of dependencies-of-dependencies-of-dependencies that bumped MSRV in a minor version. Even today, with all its flaws, minimal-versions works better for users of old Rust compilers than the normal dependency resolution.

[untitaker]

First of all, the crates that need to be yanked are mainly from <=2015, and don't work with Rust 1.x at all. They're not crates for what we call Rust, but crates for a previous experimental unstable language that predated Rust 1.0.0 release.

That's not really what's happening in the most recently filed issues, where crates are considered for yanking if they don't work with latest Rust.

What about users of very old Rust who don't have a lockfile? They need the minimal-versions feature!

Can they use it though, since all the crates are yanked now?

[kornelski]

Yanking doesn't delete crates, only hides them. Users who used yanked crates can continue to use them, because crate versions locked in lockfiles are usable indefinitely, even if they're yanked.

[untitaker]

Of course, but I'm talking about people who use very old Rust and don't have a lockfile. They are less likely to be able to use any crate successfully as it has been yanked. Even if they manually pick the last supported version and "pin" it in their toml, it won't work.

[kornelski]

Users of old Rust who don't have a lockfile are unable to use crates.io any more. At all.

[kornelski]

I've computed MSRV for every version of every Rust crate. I've compiled all crates with all Rust versions after 1.18 (that's the oldest version I checked, because older Rust versions don't have JSON message output, are slow, and there are almost no crates compatible with 1.18, and there aren't even many working with anything before 1.31).

The results are bleak. Rust versions older than 3 releases start getting problematic. 10 releases back is barely usable if you carefully groom a lockfile. 1.31 is the oldest version that has any imaginable utility.

https://lib.rs/stats#rustc

Dependencies have a very strong network effect. It's not unusual for a project to have 100 transitive dependencies, and even if only one of them is incompatible, the whole project breaks. If you look at the stats above, you can see that most crates are broken by one of their dependencies. The other compatibility cliff is the edition, which currently sets 1.31 (Dec 2018) as the minimum Rust version worth considering. Due to the network effect and crates upgrading edition, even Rust 1.55 is becoming too old. Currently 8-10% of crates.io doesn't work with Rust 1.55 or older. If you look at actively maintained crates, it's closer to 30% of crates breaking on 1.55. So minimal-versions is good for users of old Rust crates, as it helps to break that network effect. And because it shipped (if you know the secret env var), fixing the ecosystem can help existing users.

[KamilaBorowska]

That's not really what's happening in the most recently filed issues, where crates are considered for yanking if they don't work with latest Rust.

• log 0.1.*, 0.2.* and 0.3.0 are pre-1.0 crates and don't compile on any 1.x release.
• getopts 0.1.0 to 0.2.16 is an interesting case. According to lib.rs the MSRV of 0.2.17 is 1.19~, however I was able to build it with Rust 1.18.0. Checking the actual MSRV is annoying because ancient Rust releases are stuck in an infinite loop when downloading crates.io-index (at least for me) - they can build programs once there is an actual Cargo.lock, but the problem is that Rust 1.0.0 isn't compatible with new lockfiles. In any case, yanking doesn't affect packages that are already in Cargo.lock.
• debug-unreachable 0.0.1 to 0.0.4 are pre-1.0.0 crates. In case of 0.0.1 to 0.0.2 they don't compile on any 1.x release, for 0.0.3 to 0.0.4 they manage to build, however the macro itself is broken.
• xml5ever 0.16.0 has the same MSRV as xml5ever 0.16.1.
• html5ever 0.25.0 has the same MSRV as html5ever 0.25.1.

[nagisa]

I have found myself wanting a -Zminimal-versions=this-crate-only. That is, it would select minimal versions only for the crates specified in Cargo.toml and dependencies of the dependencies would still stay most recent.

This kind of option can work quite well for determining which version specifications in Cargo.toml are invalid for the crate which I'm working on at the time (though not entirely fool-proof, dependencies' reexport doesn't get covered in particular.)

[Arnavion]

Yes, that was discussed in the comments that GH is auto-hiding.

[kornelski]

I've been compiling crates with -Z minimal-versions for testing MSRV, and once I've worked around it pulling the worst offenders:

• bitflags < 1.1.0
• cc < 1.0.43
• cmake < 0.1.42
• encoding < 0.2.33
• libc < 0.2.66
• log < 0.3.8
• mime < 0.2.6
• miniz-sys < 0.1.10
• pkg-config < 0.3.14
• rand < 0.3.23
• rustc-serialize < 0.3.24
• time < 0.1.39
• void < 1.0.2
• winapi < 0.2.8

then it works reasonably well. The versions above that I've used as a cut-off for too-old are newer than absolute minimum, but I still think it's mostly a matter of yanking broken pre-Rust-1.0 crates, and the option will be useful enough to stabilize.

The rest of breakage is mostly due to serde and cc adding new features without bumping semver-minor version, and users using serde = "1", but that is good — it is actually finding real problems of specifying too-old versions or too-new features.

[epage]

I also added to my list in https://github.com/rust-lang/libs-team/issues/72:

Stablize #[stable] and have rustc produce warnings when API items with a #[stable] are used that are newer than the oldest compatible version

which should help catch these problems earlier

The versions above that I've used as a cut-off for too-old are newer than absolute minimum, but I still think it's mostly a matter of yanking broken pre-Rust-1.0 crates, and the option will be useful enough to stabilize.

I don't know about others but I still think this feature requires too much cooperation among the ecosystem as-is and that we should modify this to only control the version for direct dependencies.

The rest of breakage is mostly due to serde and cc adding new features without bumping semver-minor version, and users using serde = "1", but that is good — it is actually finding real problems of specifying too-old versions or too-new features.

I can't see how bumping patch rather than minor has anything to do with this. Regardless of which field is bumped, its a problem with whether the version requirement is bumped when new features are used.

[kornelski]

Warning about mismatch between rust-version and #[stable] is going to be useful, but it's tangential to minimal-versions. minimal-versions is about mismatch between features and versions of crates, not features and versions of Rust itself.

I keep bringing up pre-Rust-1.0 crates, because they break builds for reasons mostly unrelated to their upstream crates. Their users may have used their public API correctly, so the build failures are false negatives. minimal-versions checks if graph edges of the dependency graph are correct, but the pre-1.0-Rust crates are leaf nodes that are broken themselves regardless of the context they're used in. It's a different kind of breakage: broken leafs break the edge-tester.

For minimal versions, bumping semver-minor instead of semver-patch is as crucial as the difference between bumping semver-minor and semver-major for maximal versions. If people don't care about the difference between major and minor versions, then updates cause breakage. When people don't care about the difference between minor and patch, then downgrades cause breakage.