MSRV policy for libc crate

[joshtriplett]

In a previous libs team meeting, we discussed MSRV policy for the libc crate, and we debated between an N-1 policy and a slightly more conservative policy such as N-3 or N-4. We ended the meeting without having settled on a specific consensus, though it seemed like we were roughly on the same page.

Rather than make ad-hoc changes based on a non-specific MSRV policy, I think we should go ahead and nail down an MSRV policy for present and future changes.

I'd like to propose the following concrete policy:

• Support the last three stable versions of rustc (stable and the prior two versions).
• Only bump MSRV when we use a feature requiring newer Rust. (We do not systematically bump MSRV each time a stable release occurs.)
• Treat MSRV increases as an ordinary change, not a breaking change.

Note that this is a policy proposal for libc, not for other crates maintained by the libs team. After we get consensus on an MSRV policy for libc, we can consider what policy we want for other crates. Nonetheless, libc MSRV policy seems likely to serve as a defacto lower-bound for supported Rust versions among other libs team crates, and among many crates in the ecosystem.

For further information on the benefits of this: the PR dropping support and version detection for older versions of rustc was able to remove ~1600 lines of code, and that's not counting subsequent work that may be able to drop some of the wrapper macros. This represents a substantial improvement to maintainability, and may make it easier in the future to move to more automation to keep libc updated with the latest platform APIs.

[the8472]

which means that 10-15% of people get cut off by a window as short as S-4

From updates. Just as they already cut off rustc updates. How often is updating libc necessary? E.g. rand still defines a minimum of 0.2.22, so it'll keep working.

[workingjubilee]

I should also note, since I mentioned 1.31, that really we should probably at least be looking at 1.36, as we should be looking at MaybeUninit:

• https://github.com/rust-lang/libc/issues/1453

[thomcc]

Honestly, regardless of whether or not 1.31 or 1.36 is the true lower (err, upper? Extremal?) bound, I think that's quite a bit more conservative than is warranted here. I think we'd need a really good reason to go past S-{9,10}.

Personally, I think S-{4,5} is probably good enough (and of those I'd lean towards 5 for a clean "6 months" -- but only for aesthetic reasons) especially if coupled with implementing support for taking rust-version into account when resolving versions, although I don't really feel like that's required for a shorter bound like this to be viable.

I also think it's worth keeping in mind that most of the reason to bump libc versions is just to get new functions, for code that doesn't need them, it's not like it offers that much of a benefit most of time. So really, this is mostly about the ecosystem impact. Sadly, as @joshtriplett points out, there are a lot of really compelling features that we might want to use in libc, so its... tenuous either way.

However, given that this is largely about the ecosystem impact, I think it's especially important that we do not choose a variant that involves bumping semver on MSRV update, as that risks telling people that this is good/expected behavior.

[kornelski]

How often is updating libc necessary? E.g. rand still defines a minimum of 0.2.22, so it'll keep working.

I’m afraid it won’t help most people. It’s difficult to make Cargo pick less than the latest version of a crate. Semver exact versions or ranges tend to cause conflicts. cargo update -p libc@0.2.22 --exact helps, but every user needs to know about it and run it. BTW,-Zminimal-versions is often broken, ironically, by libc 0.1.

[Lokathor]

If cargo respected rust-version as a way to cap what version to use that'd be great. However, even once that's added to cargo, you need to allow quite a time until that version of cargo made it into linux distros. Because as much as it's simple to say "people shouldn't use a distro's cargo/rustc and expect crates.io stuff to build!", that's a really insider distinction to draw that particularly hurts the newcomers experience.

there are a lot of really compelling features that we might want to use in libc, so its... tenuous either way.

All of the "really compelling features" will still be making their way into libc, just a little tiny bit later is all. With all that happens in rust all of the time, I bet you won't even notice that much if the delay to add stuff to libc is a little longer instead of a little shorter.

[thomcc]

However, even once that's added to cargo, you need to allow quite a time until that version of cargo made it into linux distros

Worth noting that the compelling feature of a respected rust-version, if implemented, would also eventually make its way into distros, just a little tiny bit later ;)

that's a really insider distinction to draw that particularly hurts the newcomers experience

This is already the case though. There's probably nothing we can do about it either, other than slowing the update schedule of Rust itself, which isn't particularly great. Anyway, rust-version even as currently implemented should (hopefully) give new users a good enough actionable message about this.

[CAD97]

I'd like to echo @workingjubilee here: if the purpose of the MSRV policy is in any part motivated by appearing stable (implicitly: to users used to a C++ support timeline) then a "flag day" policy along the lines of "the first version supporting the previous edition" is perhaps better than a rolling MSRV.

This is due at least in part to the C++ release timeline being a 3-year cadence; C++11, C++14, C++17, C++20, C++23, etc. The "stable like C++" viewpoint is used to "flag day" upgrades for requiring support of new C++ standards.

This is, of course, counter to the goal of Rust's release trains of getting new features out when they're ready, and puts unwanted flag day pressure on "the edition release." But a rolling support window is a new thing you have to convince a C++-biased ecosystem to accept, when they're used to a flag day LTS style support model.

What this does suggest though, is interesting: just because you support e.g. Rust 1.31 (release of edition2018) doesn't mean you support every version between that and stable. It would be a completely reasonable policy for libc to only provide support for use on the 1.31, 1.56, and stable-N toolchains, and remove any version sniffing finer than that.

(This is not to say that in-between versions would cease to work; because of language stability they would. What wouldn't necessarily be available are the progressive feature updates between LTS toolchains.)

However, taking on such an LTS policy would probably require coordination with the (language? and) whole ecosystem to agree on what toolchain to provide library LTS for.

---

So how to draw an actual policy from this? I honestly don't know. "first version of the previous edition" is definitely an upper bound for LTS. A rolling interpretation of that would be a 3 year support window. A flagged version of exactly that is probably too infrequent[^1] for the pace of Rust evolution.

[^1]: And almost certainly will be until there exists a validated compiler (e.g. via ferrocene) that it makes sense to keep longer support for.

Using Debian as a baseline for expectations would suggest specifying a new "library LTS target" every two years and supporting the previous target for a year after the new one is available. Interestingly, this actually lines up somewhat well with a rolling 3-year support window.

Massaging that a bit to better fit into the Rust edition cadence suggests a new "library LTS target" every year and a half (e.g. the first release of the edition and the release 12 stables (72 weeks / 16½ months) after it) and to drop support for the old LTS target ~9 months (or more likely 6 stables / 36 weeks / 8¼ months) after the new one is available. Normalizing the edition release cadence to every 24 releases (or perhaps 26) rather than "every three years" would assist in such a scheme.

At any given time, the MSRV under such a policy is between stable-5 and stable-17. If you immediately drop oldlts then it's between stable-0 and stable-11. ("zero indexed")

I'm not confident in this, nor am I actually proposing it. But if the goal is (in any part) to fit into a C++-colored view of stability, a scheme like this seems desirable moreso than a rolling window.

(LTS targets every 6, drop support after 9, editions every 24 seems reasonable and seems to fit with the "six months" temperature. This would vary between stable-3 and stable-8.)

[the8472]

If cargo respected rust-version as a way to cap what version to use that'd be great.

cargo update -p libc@0.2.22 --exact helps, but every user needs to know about it and run it.

A middle ground would be teaching cargo to print a hint about --precise as one approach among several to resolve the problem.

[riking]

Anything less than 1 year is at high risk of being unworkable for your average software-using corporation. Toolchain updates just aren't a high priority unless given specific impulse, and a consistent calendar date makes it easy to have an upgrade cadence in your yearly planning.

For some real-world data here, Kubernetes attempts to declare a 1-year support window. If you look at what the major cloud providers actually manage to support, real-world usage trails that by about 3-4 months (or worse when there's a particularly burdensome upgrade). And Kubernetes has a strong reputation for being unstable and having high "infinite treadmill" costs to use, so it's hard to justify anything more aggressive.

[joshtriplett]

@thomcc wrote:

Honestly, regardless of whether or not 1.31 or 1.36 is the true lower (err, upper? Extremal?) bound, I think that's quite a bit more conservative than is warranted here. I think we'd need a really good reason to go past S-{9,10}.

I think we'd need a really good reason to go past S-{4,5}.

@CAD97 wrote:

if the purpose of the MSRV policy is in any part motivated by appearing stable (implicitly: to users used to a C++ support timeline) if the goal is (in any part) to fit into a C++-colored view of stability

Emphatically not. We should be looking to support Rust users and Rust user needs.

Perhaps someone who feels motivated to maintain an "LTS version" of Rust based on multi-year timelines might also be interested in maintaining a crates.io mirror that automatically lags by omitting any crates declaring a rust-version newer than that LTS, or anything that depends on those, recursively. That would place the maintenance burden more appropriately.

What this does suggest though, is interesting: just because you support e.g. Rust 1.31 (release of edition2018) doesn't mean you support every version between that and stable. It would be a completely reasonable policy for libc to only provide support for use on the 1.31, 1.56, and stable-N toolchains, and remove any version sniffing finer than that.

That would not make maintenance any simpler. The maintenance burden arises from needing to support less capable versions at all, not from the number of such versions. Rust versions have strictly increasing feature sets, so it's really not meaningful to support 1.31 and not support 1.32.

@Lokathor wrote:

I bet you won't even notice that much if the delay to add stuff to libc is a little longer instead of a little shorter.

Yes, we really will. We add features to Rust with the intention of using them, often because people are experiencing issues due to the lack of those features.

@workingjubilee wrote:

I threw out the "we should at least not ask for more than 1 Edition behind" partly because I think we should at least be putting forward some lower bounds for what we're willing to consider

I agree. For my part, while I originally proposed N-2, I could live with N-4 or N-5 if absolutely necessary, and would start vigorously complaining if new features need to wait longer than six months. "stable distro" support is several times longer than I'd ever want to consider.

[joshtriplett]

@thomcc wrote:

which means that 10-15% of people get cut off by a window as short as S-4

From updates. Just as they already cut off rustc updates. How often is updating libc necessary? E.g. rand still defines a minimum of 0.2.22, so it'll keep working.

Exactly. An MSRV policy does not mean that users on older versions are completely unsupported; it means that users running older rust also need to run older crates.

@the8472 wrote:

cargo update -p libc@0.2.22 --exact helps, but every user needs to know about it and run it.

A middle ground would be teaching cargo to print a hint about --precise as one approach among several to resolve the problem.

This is an excellent idea, and I'm going to write a PR for it right now.

If we also had rust-version information in the index (which we currently don't), we could even say something like "you need to upgrade to Rust 1.xy, or run cargo update --precise -p crate@ver as ver is the most recent compatible version supporting Rust 1.yourversion".

[algesten]

I have a variant of "My company pins its Rust version in CI and it takes a lot of resources to upgrade." that maybe hasn't been entirely explored here with regards to official Docker versions of Rust.

At work, one of our projects integrates quite deeply into some (bespoke) C-code. That code in turn has transitive dependencies on some libraries that are mostly installed with apt-get. For deployment we build in CI using Docker – specifically the official Rust docker images + apt get on top.

As it happened, some of that C-code was dependent on specific library versions bundled with Debian Stretch and the last official Rust Docker image for Stretch was Rust 1.45. We didn't have time to fix all this code at the time, but with each compilation something else broke. So we ended up gradually pinning project by project – effectively held captive at 1.45 for about 1 year before we got around to it. (Situations like this ultimately can become a commercial liability due to enterprise contracts that require us to maintain certain update policies. Ultimately you just "have to fix it", when the risk of those contracts starts to outweigh whatever benefit you gain from living with the problem)

Painful. Totally our own mess. libc NOT shifting under our feet during this time was likely a blessing.

No idea how common our case is – maybe not worth considering. In my mind it illustrates:

• Official Docker images have another cadence where they shift from one base image (Debian Stretch) to the next (Debian Buster), and can in cases like this introduce "cliff edge drops" between seemingly insignificant Rust versions (not tied to editions), i.e. 1.45 - 1.46.
• Rust has excellent C interop which is actively promoted as a benefit of the language. In projects that mix Rust and C, there might be an additional layer of expectation that it's possible to adopt a more C style minimum version policy.

And on a personal note:

• Given that the libc MSRV is likely to set the tone for the ecosystem, I my above situation, would feel way better with a more conservative at least N-9 (12 months) policy.
• There's some dissonance between Rust stability guarantee, of complete backwards compatibility on the 1.x branch – and a libc MSRV that promises 6 months. I know they are not the same, different teams etc. It seems that to arrive at these polar opposites, they can't have started from the same place... "forever" vs "6 months"

[CAD97]

There's some dissonance between Rust stability guarantee, of complete backwards compatibility on the 1.x branch – and a libc MSRV that promises 6 months. I know they are not the same, different teams etc. It seems that to arrive at these polar opposites, they can't have started from the same place... "forever" vs "6 months"

Just to clarify: these are highly different. libc has the same semver guarantee that the Rust language does: that you can upgrade from { libc 0.2.x | Rust 1.x } to { libc 0.2.y | Rust 1.y } and everything will continue to work.

Rust does not guarantee that Rust 1.y will continue to support any given OS version "forever". In fact, the Rust project has dropped support for old OS versions previously (e.g. Windows Vista) and is discussing the timeline for dropping support of other legacy out-of-support OS versions (e.g. Windows 7).

The equivalence of policies would be that libc's Rust toolchain support parallels the policy for Rust target support. I.e. it would be consistent with the Rust language target policy for libc to not support being compiled with any out-of-support toolchain, i.e. any toolchain which is not the latest stable.

[saethlin]

official Docker versions of Rust

I really worry about these. They have been the cause of some issues at my current employer already because people seem to have a strange attachment to them. I don't know why. They're just a base image with rustup and a default toolchain installed. I'm slowly trying to steer people away from them because I think the "official" gives entirely the wrong impression.

There's some dissonance between Rust stability guarantee

The Rust stability guarantee is about using a new compiler with old code, not using an old compiler with new code.

[joshtriplett]

@algesten wrote:

Painful. Totally our own mess. libc NOT shifting under our feet during this time was likely a blessing.

No idea how common our case is – maybe not worth considering.

FWIW, it's absolutely worth considering, and thank you for raising it.

Whether we end up picking a point on the spectrum of tradeoffs that addresses it is another question, but whether we do or not, it's absolutely worth considering and understanding.

There's some dissonance between Rust stability guarantee, of complete backwards compatibility on the 1.x branch – and a libc MSRV that promises 6 months. I know they are not the same, different teams etc. It seems that to arrive at these polar opposites, they can't have started from the same place... "forever" vs "6 months"

"forever" is the support timeline for "you can compile old crates on a new compiler". It's critically important that we maintain that, so that people can confidently upgrade Rust. That is one of the major reasons why we can be more comfortable saying "you should upgrade to a newer rustc": we do not break people's code.

What we're talking about here is a support timeline for "you can compile new crates on an old compiler". That's an entirely different matter, with different tradeoffs and different requirements.

[workingjubilee]

Rust's stability guarantees are inversely shaped to the (highly relative!) stability offered by most C or C++ ecosystems. The promise is, roughly, "The code you write today will, if it is built on the solid foundations of Safe Rust using stable features, continue to compile, and we only will change this if it absolutely has to be fixed."

That is, the promise is that Rust 1.999999999.0 should still compile today's libc. If it doesn't it is because that future version of the Rust compiler will have somehow validated that code as actually unsound, a mistake that should never have been admitted into the Rust language, etc. etc., and then and only then rejected it, on the premise that it would have been miscompiled anyways. In practice this should happen Approximately Never but maybe by then the Rust compiler will magically detect that one of our bindings against the C interfaces is actually written in an unsound way and it's a miracle that it compiled at all.

C code instead biases towards "still compiles with C89". This is a very, very different animal, which asserts essentially that code that uses new features in C (yes, they exist!) functionally does not necessarily exist.

[joshtriplett]

@workingjubilee Indeed. Projects still sometimes get bug reports complaining that they require C11. That's not something we should aspire to emulate or encourage.

[Lokathor]

Well no one is suggesting S-95 ;3

But S-10, possibly even using features to opt in to newer compiler versions sooner(!), would seem to help a number of people, and all the updates would still come to libc in time, just with a little extra delay.

If libc were a "nearly totally internal" crate like stdarch or compiler-builtins or libm then I'd be with you that we should throw in the new stuff just about as soon as it comes out. However, libc is an extremely user facing crate, so a bigger time gap seems warranted.

[algesten]

What we're talking about here is a support timeline for "you can compile new crates on an old compiler". That's an entirely different matter, with different tradeoffs and different requirements.

I'm totally with you that there is a big difference between the stability guarantee and this issue, but from a user's perspective I argue they are in the general realm of each other. Situations where code can "just stop compiling" undermines the stability guarantee.

C code instead biases towards "still compiles with C89". This is a very, very different animal, which asserts essentially that code that uses new features in C (yes, they exist!) functionally does not necessarily exist.

Projects still sometimes get bug reports complaining that they require C11. That's not something we should aspire to emulate or encourage.

I can see the appeal of the 10+ years horizon some of these libraries, but that's extreme. 6 months seems to me like a leap to the extreme in the other direction, which is why I argue for a compromise of something like 12 months.

[mitsuhiko]

Generally I think between 6 months stability and just current and previous Rust compiler stability there is really no difference in practice. If I have a six month old compiler I'm likely already in a workflow where I am upgrading regularly.

[cuviper]

@joshtriplett

I think we'd need a really good reason to go past S-{4,5}.

Please consider a year. I already outlined how our update timelines work for Rust in RHEL, which is relatively aggressive for an enterprise distro, but a ~6 month window will still leave that behind. If you don't consider this a good reason, then I fear you're implying that Rust is not suitable for such environments at all.

@CAD97

The only officially supported configuration for compiling Rust code is using the latest stable toolchain. So if you're compiling not on the latest stable, you're using an unsupported configuration.

That's the only supported configuration by The Rust Project, but RHEL's rustc is supported by Red Hat, and even if you found that lacking you could contract support elsewhere, because this is open source. There doesn't have to be an upstream-blessed LTS for these kind of arrangements.

Plus, if you tell people that they can only compile for RHEL, SLES, etc. if they use the rustup-provided toolchain, then the project also ends up being the front line in supporting every such user. But if you give some room for those intermediaries, then they can shield some of that.

[joshtriplett]

@cuviper Rust is absolutely suitable for enterprise Linux environments, as long as the same environment providing the toolchain also provides the crates.

If people are obtaining crates from crates.io, then obtaining the toolchain from rust-lang.org doesn't seem substantively different. (I'm also guessing that RHEL doesn't package all the cross-compile targets people might want, in which case people would need rustup for those as well.)

And if people are vendoring crates from crates.io, then that vendoring process could select crates suitable for a given version of Cargo. (Perhaps we could teach cargo vendor to warn if the vendored crate won't actually compile with the current cargo/rustc, and teach it to obtain an older version that will.)

Plus, if you tell people that they can only compile for RHEL, SLES, etc. if they use the rustup-provided toolchain, then the project also ends up being the front line in supporting every such user.

I would absolutely expect that, and I don't think that's an issue. I would never expect distributions to support a toolchain, or crate, that they didn't supply.

Does RHEL have a packaged version of rustup?

[8573]

If I understand correctly, the intended outcome is, or includes, that the libc crate maintainers' support for distro-grade rustcs will be replaced with

1. distro package maintainers' providing old versions of the libc crate that support their distros' rustcs, presumably with the distro maintainers backporting fixes from new versions of the libc crate as they do for rustc.

I apologize for the extent to which I speak ignorantly here, but have the following alternatives been considered (I can't see that they have)?

2. Distro package maintainers (maybe together with Tokio) maintain a fairly up-to-date version of the libc crate with the version sniffing code that upstream discards.

3. Distro package maintainers (maybe together with Tokio) help maintain the libc crate upstream with support for their distros, if they can provide enough assistance to offset the cost to the existing libc crate maintainers.

[cuviper]

Rust is absolutely suitable for enterprise Linux environments, as long as the same environment providing the toolchain also provides the crates.

I don't think that's tenable. The distro will certainly package the crates it needs itself in some form (Fedora has distinct crate packages, while RHEL has been vendoring), but we can't realistically package every crate that our users might want. I don't think that should be the expectation either, as for any other language you can reasonably bring your own code and external dependencies. People should be able to use system toolchains more broadly, just like rustc leans on /usr/bin/cc.

(I'm also guessing that RHEL doesn't package all the cross-compile targets people might want, in which case people would need rustup for those as well.)

I don't want to get too narrow about RHEL specifically, even though that's the target I care about... but no RHEL doesn't really deal in cross-compiling at all. If that's your goal, you already need your own sysroot and everything, so sure, get your own rustc installation for that as well.

And if people are vendoring crates from crates.io, then that vendoring process could select crates suitable for a given version of Cargo. (Perhaps we could teach cargo vendor to warn if the vendored crate won't actually compile with the current cargo/rustc, and teach it to obtain an older version that will.)

They may be vendoring, or they may be building in CI that pulls from crates.io, or they may be developers working locally on code that's meant to eventually deploy for RHEL (or similar). Still, "teach it to obtain an older version" sounds like making the Cargo resolver using rust-version, which we all agree would be great! (I'm not sure it solves everything if we get a fancier resolver and everyone starts bumping MSRV aggressively, but it will certainly help.)

Does RHEL have a packaged version of rustup?

No, and I don't see the point of that. If you're going to download the toolchain, you might as well start from rustup.rs too. And for the customer that wants to pay for toolchain support, they wouldn't be happy if we can only say "we downloaded and installed it successfully, go ask the upstream why it's not working."

3. Distro package maintainers (maybe together with Tokio) help maintain the libc crate upstream with support for their distros, if they can provide enough assistance to offset the cost to the existing libc crate maintainers.

I'm absolutely willing to help with maintaining compat support, but we have to agree that it's an acceptable goal first.

[carbotaniuman]

Another use-case for an older MSRV is dynamic linking/plugins, and Rust's lack of ABI stability.

One of the things I work on heavily uses std (and libc, but that's not the issue) types across API boundaries, with non ABI stable types behind pointer indirection, letting us do a plugin system. This means we use relatively new libraries, but are locked to an older (in this case 1.47) rustc.

I expect that this will eventually want to move to something like lccc's stable ABI / repr(Rust2021), but given that those are not yet ready, pinning rustc is our only option.

[thomcc]

That's an interesting use case that (I think) hadn't been considered, but to be honest I think you'd be better off not updating any of your dependencies either if you do that (I guess that might be hard to arrange though).

Like, you should really be ensuring all versions of all deps are identical too -- even under pretty strict interpretation of semver, it's totally fine for someone to change the ABI of a struct in a semver-patch update. For example -- reordering the fields of a #[repr(Rust)] struct for example can change the ABI... let alone more interesting ways of that breaking...

So, while libc might be one of the only deps you can get away with this on at the moment, it's still... not ideal. And I'm not sure it's something we care much about supporting -- certainly in so many other ways it's clearly not something that's really supported.

[yerke]

I wanted to raise one point that I don't think was brought up before. Sometimes it's painful to upgrade the compiler to the latest stable due to the breakages in the compiler itself. For example, the incremental compilation was broken on 1.52, was disabled in 1.52.1, and remained disabled in 1.53 until being re-enabled in 1.54. I do understand that it's a one off incident, but something similar could potentially happen in the future. I also realize that the official recommendation was to still upgrade to the corresponding latest stables, but I am pretty sure some people/companies didn't follow that advice.

In the example above, I think following latest stable - 2 would have been a bit painful.

To be clear: I personally like to upgrade to the latest stable pretty much immediately. At work (where we admittedly don't use much Rust) I have not experienced any issues upgrading the compiler either in CI or on developer machines. We use CentOS 7, but in the project I am involved in, we have no problem installing rustup in Dockerfile and then installing the compiler version we want, when building binaries in CI.

Re the rust compiler version provided by the distros (with the important caveat in that I don't develop on Linux, but rather on macOS, but I do target Linux on prod, and so I don't have a lot of experience with Linux). If some people/companies do not want to use rustup, are they comfortable to use rustc versions from their respective package managers? For example, right now people with Ubuntu with versions starting from 18.04 can install 1.59 with just sudo apt install cargo. For Ubuntu 16.04 the similar way got me to 1.47. So 1.34, as suggested by some people, is way too conservative IMO.

[Kixunil]

@joshtriplett the issue with "use distribution-packaged crates" is that cargo doesn't work with them out-of-the box. Manual [patch] is annoying and disturbing to users who want to use crates from crates.io. If cargo was modified to support it somehow that'd be really nice and perhaps the solution but from my short thinking it seems quite complicated issue. Python seems better here since it already does read system libraries and works well in my experience but explicit installation is a bit annoying.

[kornelski]

@yerke My understanding is that incremental compilation has been broken in earlier Rust versions too, so if you did not upgrade, you would have broken incremental compilation. The bug has been discovered around 1.52 and then mitigated (by disabling) because it was known, but it has existed before in earlier Rust versions. So there was no rational reason to stay on an older compiler. Plus, there was an env flag to re-enable old compiler's buggy behavior if you really wanted to upgrade Rust but keep existing incremental compilation.

[Noah-Kennedy]

Been doing some thinking around this.

I think it's quite reasonable libraries to maintain an MSRV by specifying their dependency semver requirements such that using the minimum allowed versions of all dependencies for the library will result in successful compilation. Using an old rust version could be treated as an "advanced use case" that requires some manual intervention on the users part to make it work.

We could also make this easier by potentially having an MSRV field in Cargo.toml and using that for a special cargo update --msrv <MSRV> command, which updates to the latest MSRV-compatible crates, although how that would work with crates that don't supply an MSRV is unclear, and would require further thought.

As a reminder, libraries cannot really use dependency pinning to solve this, as trying to bring in both libc version v0.2.126 and v0.2.125 into a dependency tree results in a compilation error.

This is one potential way to go about solving this type of issue, however it brings up a lot of issues around how most rust packages are developed. They tend to be written often selecting the newest available rather than the newest compatible versions of crates, which makes any attempt to use older crates (even if you keep them vendored to bring in security patches) very annoying. This is actually one of the issues as well that IMO vendoring cannot be a solution here, as maintaining a vendored libc (or any other crate) doesn't mean that your crate will work with the rest of the ecosystem. The only thing you can really do here with vendoring is backport patches to older versions.

[epage]

@Noah-Kennedy

by potentially having an MSRV field in Cargo.toml

You are in luck, work has already started on this! See rust-version

using that for a special cargo update --msrv command, which updates to the latest MSRV-compatible crate

https://github.com/rust-lang/cargo/issues/9930 is the issue for this. https://github.com/dependabot/dependabot-core/issues/5423 is the parallel issue for Depandabot.

Considering one is a big lift (for the full UX we want) and the other is dependent on Github, they are unlikely to be resolved immediately. Until those are implemented, we have

• https://github.com/rust-lang/cargo/pull/10902 to teach people how to downgrade dependencies
• https://github.com/rust-lang/cargo/issues/10903 to cover a hole left in the above PR
• https://github.com/rust-lang/cargo/issues/10653 to ensure version reqs are likely to be compatible so people can downgrade

It could possibly also help if we change our recommendations to committing Cargo.lock for libs so cargo update --precise changes are preserved.

as maintaining a vendored libc (or any other crate) doesn't mean that your crate will work with the rest of the ecosystem.

A big help for libc on this front is upgrading the MSRV

@joshtriplett said

The primary issue with libc major version differences would be the c_* types. However, we can solve those in two different ways: have the older version re-export the newer, or have them reuse the types now available in core::ffi. Both of those, however, would result in a dependency on newer Rust.

[Noah-Kennedy]

Ah, I totally misunderstood what that field was intended to be, excellent! Glad to here that work has started!

[joshtriplett]

This was unintentionally labeled T-libs-api rather than T-libs; fixing that.

@rfcbot cancel

[joshtriplett]

I'm going to refrain from starting another FCP until we make some further progress on the discussion.

[ChrisDenton]

So this thread is quite long, conversation has spread to multiple places, and I feel that the arguments are maybe focused on the "why not" and relatively little has been said about "why". Note that here I'm summarizing what other people have said but these are my own words. Any mistakes or misrepresentations are my own so feel free to correct me. Summarizing theses points should not be construed as me either agreeing or disagreeing with them.

The benefits of a shorter MSRV are:

• It allows removing compatibility shims. These can be a lot of code with a high maintenance burden.
• libc can potentially adopt stabilized features faster. This may not be needed very often but when it is then waiting years may be painful,
• Supporting older versions is extra (often unpaid) work for maintainers and contributors
• rust-lang itself only supports latest stable rustc. Other organisations can and do support older versions. The users of these supported older rustc should look to the organisation to support their workflow rather than pushing the cost of support on to library authors (see also the previous point).

[joshtriplett]

@ChrisDenton I think all of those are accurate, and I'd entirely agree with all of them. There are a few more I would add:

• Support for older versions is not actually particularly well-tested or correct; the effective API surface area of libc for older versions is in some cases just blatantly wrong.
• There's a multiplicative complexity in libc, where we already have a substantial support burden of handling all the different targets, and supporting all the targets on all the versions of Rust makes for even more complexity.
• Support for older versions will substantially complicate attempting to automate libc API generation. (This is an issue with new features, not just existing ones.)

[cuviper]

• Supporting older versions is extra (often unpaid) work for maintainers and contributors
• rust-lang itself only supports latest stable rustc. Other organisations can and do support older versions. The users of these supported older rustc should look to the organisation to support their workflow rather than pushing the cost of support on to library authors (see also the previous point).

Project willing, that maintenance work can also be shared amongst those organizations in the upstream repo when a longer MSRV is chosen. This could be sort of similar to the target-tier policy which requires active investment for less common targets. As I've said, I'm very willing to do such work on Red Hat's time, especially as it pertains to rust-lang crates. I may not see when folks are struggling, but you can tag me if you want help.

[rfcbot]

@joshtriplett proposal cancelled.

[bstrie]

Some prior arguments have been made that having an old MSRV increases the perception that "Rust is stable". This is an overloaded statement that I want to unpack and elaborate on:

1. To some people, "Rust is stable" means that the language isn't making breaking changes. In practice, Rust is up there with the most stable languages around, as shown by Red Hat's confidence in shipping (relatively) rapid toolchain updates.
2. To some people, "Rust is stable" means that Rust's public library ecosystem is mature enough that it adheres to semver, with library version numbers accurately reflecting the existence of breaking changes. In practice, Rust is quite good at this: IME you are unlikely to ever hit a backwards-incompatible change from an automatic semver-compatible version update, even from 0.x crates.
3. To some people, "Rust is stable" means that Rust's public library ecosystem is predominantly composed of crates that are version 1.0 or greater. In practice, Rust is middling at this, but improving over time.
4. To some people, "Rust is stable" means that Rust's public library ecosystem is moving slowly enough that new versions of crates do not tend to require extremely recent toolchains to build. In practice, Rust is quite far from this.

For the purpose of this discussion, the fourth population of users are the ones to whom Rust appears unstable. Many of these users may be basing their expectations on the C and C++ library ecosystems, which tend to have this "stable" property. To argue that Rust's ecosystem should adhere to a very old MSRV is thus an argument aimed at making Rust more attractive to this fourth population.

However, I believe this logic is precisely backwards. Rust's ecosystem isn't unstable because it don't enforce an old MSRV; Rust's ecosystem doesn't enforce an old MSRV because it's unstable! Or to use a less loaded term than "unstable", Rust's ecosystem doesn't enforce an MSRV because it's moving very fast, and it's moving very fast because it's very, very young in the grand scheme of things. Seven years since 1.0 may seem like an eternity to some of us, but C's ecosystem wasn't nailed down as of 1979, nor was C++'s in 1992, nor was Python's in 1998. Ultimately it's not possible to use MSRV policies to make the Rust ecosystem appear more stable than it actually is; the only remedy for that is time. If that deters some potential users of Rust, then that's a perfectly acceptable conclusion for them to draw and we can hope that they'll reevaluate in 5-10 years. In the meantime, let's be willing to act like the young ecosystem we are, while at the same time of course not using youth as an excuse to be irresponsible to users with reasonable objections to frequent upgrades.

---

Personally, with a crate as uniquely foundational as libc, I think it's a show of good faith to support Rust versions as old as 1.5 years; that's N-12[^1]. For crates that may be foundational in more specific domains, (e.g. Serde), I think encouraging at least a year-old MSRV would be fairly generous; that's N-8. For crates that tend to appear only shallowly in dependency trees, I think it would be conscientious to offer an MSRV of N-4[^2]; since every other major language on the planet only issues feature-based releases at most every six months, even that level of restraint would make us as rapid as every other language can possibly be.

[^1]: Arguably crates as foundational as this have a strong argument for just being in libstd in the first place, which is why I was so happy to see the recent expansion of core::ffi (thanks, Josh!) [^2]: The delightfully informative pip data from above does tend to show a relatively large, nonlinear jump between the averaged marginal availability of N-2 and N-3, suggesting that a sizeable segment of users don't want to update more than three-ish times a year

[Noah-Kennedy]

I think one thing that is worth pointing out here is that compilers are like any other software in that code churn from new features produces new bugs, and sometimes these bugs can go a while before they are patched. This is one of the core reasons why enterprises often rely on pinned version. They know that the version they have tested their software around works for what they are doing, and they may be reluctant to move to a revision with new bugs which requires significant fresh QA work to verify their software on.

The point I'm trying to make here is that this is actually fairly normal, and there are major companies who use Rust this way because developing reliable software is hard, and if your software needs to meet standards of reliability, you need to look at your compiler as another source of bugs and issues, and look at upgrading it as a task which brings real risk.

[thomcc]

They should also avoid updating dependencies, as the same logic applies there as well.

The compiler should actually be better than most code on crates.io as it has thorough test suites and every potential release gets evaluated for multiple months before reaching stable.

[Noah-Kennedy]

Yes, but those are easier to upgrade incrementally. If you take the approach of upgrading when there is a vuln or when you actually need a new feature, you don't upgrade your crates as often. And some libraries are easier than others to upgrade. libc is very low risk for example, and can be upgraded fairly easily.

[Noah-Kennedy]

@thomcc this is true as well that it is more tested than most crates, but also, compilers are massive code basis, with far more complexity and surface area than typical libraries.

[workingjubilee]

@Noah-Kennedy In practice, of issues opened against rust-lang/rust, at the current moment:

• 648 are marked regression-from-stable-to-nightly
• 730 are marked regression-from-stable-to-beta.
• 754 are marked regression-from-stable-to-stable.

Of those

• 644 issues were closed as nightly regressions (~99.4%)
• 723 issues were closed as beta regressions (~99.0%)
• 604 issues were closed as stable regressions (~80.1%)

Now, some of those unresolved stable-to-stable issues are ones that have slipped from beta to nightly to stable, but the hit rate is so much remarkably worse, and new stable-to-stable regressions do crop up over time, that with Rust, I must disagree with the conclusion that "sometimes compiler bugs aren't fixed immediately, therefore pin versions":

Best practice would actually be to test with the beta and nightly compiler to surface regressions, and to deliver software from a stable compiler. And because 1.X.Y stable is almost always 1.X.0, and there is very rarely a patch release, to keep marching the stable compiler forward as often as is reasonable. The alternative is missing out on security fixes, and the fixes to those regressions, also, including stable-to-stable regressions that predate your current rustc version. If you aren't backporting fixes, there's not much point to pinning your rustc version, unless it's a critical build regression (and of course, sometimes it is!).

And I have observed enough of such conversations to know that yes, people do talk significantly differently about catching beta and nightly regressions versus stable-to-stable regressions. Stable regressions are desirable to fix, but it is generally something to Eventually fix, there is... less of a need to hurry. So if an enterprise is waiting until it is a stable-to-stable regression to report it, even when they have the resources to add a couple extra CI jobs, and not trying to catch issues as close to the edge as possible, then that enterprise is making sure that the bug is not fixed for a while.

Many of the currently standing regressions are small missed performance optimizations, things like doing a comparison twice for some reason, even though ground has been gained in other areas. Rarely are they regressions in soundness. One of the reasons that compilers for other languages produce regressions is precisely because they rely on things like undefined behavior as the basis for performance optimizations. An occasional missed opportunity to prune a provably removable bounds check is as nothing before the kinds of optimization that true undefined behavior can justify, making programs go sideways. It's not that we have no bugs, but we have a much different distribution of them.

[CAD97]

@Noah-Kennedy It's also worth noting another big reason upgrading rustc is much different than upgrading e.g. gcc:

Approximately zero C++ programs are free of UB. Most Rust programs are free of UB.

Upgrading a compiler means you're at risk of turning "benign" UB into a "miscompilation" and getting undesirable results where the previous compiler did the correct thing. This is the risk of using an optimizing compiler with a language that offers zero safeguards from writing code the compiler does not have to preserve the semantics of (i.e. that has UB).

This alone means that upgrading rustc is much less likely to break your program than upgrading a C++ program. If upgrading a C++ compiler "miscompiles" your C++ program, there's approximately a 70% chance[^1] (if not more) that it's due to your program having UB and the compiler only now taking advantage of it in a way that breaks your program.

[^1]: Yes, I'm misusing that statistic, but if anything, the proper number is higher, not lower.

The primary advantage of using Rust over C++ (for C++ users) is memory safety and a lack of easy-to-hit UB. As a direct consequence, upgrading the compiler also becomes a lot less likely to cause miscompilations. Most Rust stable-stable regressions are in performance (not correctness/soundness) or in the compiler not accepting input it used to (e.g. due to ICE).

And adding to that, even if upgrading rustc were anywhere near as risky as upgrading a C++ compiler, it'd be maximally the same risk over the same time period. So it's not the risk of a 60-week (14 month) C++ compiler upgrade every 6 weeks; it's the risk of a 60-week C++ compiler upgrade spread out over 10 chunks every 6 weeks.

As such, (despite providing arguments and suggestions for a longer support timeline upthread,) I vehemently agree with @workingjubilee that it is absolutely better to regularly upgrade your Rust toolchain without any special testing, just with your normal staging smoke tests you're doing anyway before deploying to production. (If your enterprise refuses to upgrade rustc regularly but deploys to production without smoke tests on staging first... they need to get their priorities straight.)

The hard part is convincing executive managers burned by C++ compiler upgrades that Rust is different. This is, I think, a significant factor of why @joshtriplett has asked multiple times for concrete user stories asks for older toolchain support — a majority of inertia for toolchain upgrades is based near exclusively in organizational inertia inherited from C++, rather than any intrinsic reasoning that applies equally to rustc. You can even make the argument that "install from the distro" is a C++-inherited constraint: distro packages of C++ code/tools are much more likely to play well together than alternative installation sources (if you can even build them), whereas rustup and cargo just work 99% of the time (and when they don't, it's typically because of C/C++ dependencies).

[djc]

One group that I think we're trying to support here is a group of users (not Rust developers) who would just like to be able to build applications (even if just by running cargo install) with the compiler provided by their system package manager, which seems entirely reasonable and not that uncommon. For comparison, in the Python ecosystem, people will often support language/interpreter versions that are more like 2-3 years old.

At risk of repeating @mitsuhiko's thoughts, I've mostly found supporting older Rust versions not to be such a problem. Yes, it can be a little frustrating that there are nice features in std or the language or Cargo that I'd like to use, but having recently ported a significant amount of code from 1.46 to 1.32 (for chrono), it wasn't that hard. (Though I'll admit to not writing a lot of FFI code so I'm not sure what specific features you'd miss there.)

As a reminder, libraries cannot really use dependency pinning to solve this, as trying to bring in both libc version v0.2.126 and v0.2.125 into a dependency tree results in a compilation error.

As I understand it, this means that the MSRV policy for the libc crate imposes a lower bound on the MSRV policies for all transitively depended library and application crates that want to use it. As such it seems to me that using something relatively conservative like N-12 is warranted. I've been frustrated recently with fairly fundamental crates like async-std and hashbrown that adopted somewhat aggressive MSRVs (1.56) because they make it impossible for me to keep the MSRV policies that I'd like to support in crates I maintain (things like chrono, indicatif, rustls).

If libc adopted a policy like N-5 which I would consider fairly aggressive (for libc), I think it would be likely that a fork happens, fracturing the ecosystem.

[8573]

Would a fork necessarily fracture the ecosystem? The fork could offer conversions between its types and the original libc crate's, behind an optional dependency on the original crate.

[kornelski]

On a technical level fork of libc wouldn't be too bad, since it's mostly just FFI definitions and constants. However, from ecosystem perspective it's likely to cause a lot of churn, and tire maintainers with requests/debates to switch to one or the other.

[kornelski]

BTW: the need to use an old Rust version, because a Linux distro has it, is a specific use-case that I hear about frequently, so I think Rust could consider doing something about this problem first: https://internals.rust-lang.org/t/rust-msrv-policy-vs-linux-distros/17074

[TheBlueMatt]

As someone who delivers software written in Rust to RHEL environments, I would really like to hear from users who are developing on RHEL using the distro rustc/cargo instead of rustup. We jump through hoops to deliberately avoid using distro packages when possible, on account of their age.

Not RHEL, but here's a specific example - I ship a bunch of software that relies on cross-language LTO. By orders of magnitude the easiest way to accomplish that is apt-get install cargo clang and build with appropriate flags. Debian has historically been really good about using the same LLVM for clang and rustc (and porting appropriate rust-originated fixes to the LLVM release). Sadly, last I heard, due to some good reasons I forgot now, Fedora does not ship a rustc that uses system LLVM, so its Debian or bust.