Open joshtriplett opened 2 years ago
@rfcbot merge
Team member @joshtriplett has proposed to merge this. The next step is review by the rest of the tagged team members:
Concerns:
Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!
See this document for info about what commands tagged team members can give me.
This is such an unreasonably short support window for a foundational crate like libc. I would expect at least couple of years of supported versions instead of N-3,4
Yeah it's a bit tough, since this impacts a lot of crates that have libc in their dep tree. I think being conservative here is worthwhile, unfortunately.
One danger here is if a lot of crates end up pinning libc versions (which they shouldn't[^1], but in practice is the tool many people reach for), those crates become incompatible to use with eachother.
[^1]: Allowing users with a lockfile to deal with it via cargo update --precise
is better.
@petrochenkov Assume, for the moment, that we aren't looking to support the use case of "run brand new crates from crates.io on a multi-year-old snapshot of rustc/cargo in an enterprise distribution". Are there any other use cases for which "couple of years" is actually a specific end-user need? Not something where crate-author asks crate-author asks crate-author based on hypotheticals, but actual end users who have an old rustc, and a need for new crates, and no better options?
My instinctual response to this is also, frankly, that N-4
is too aggressive. But:
Are there any other use cases for which "couple of years" is actually a specific end-user need? Not something where crate-author asks crate-author asks crate-author based on hypotheticals, but actual end users who have an old rustc, and a need for new crates, and no better options?
I basically don't have a great concrete answer to this. And I will say, I've personally been looking for one for a while. Any time someone tells me about wanting crates to compile with older versions of Rust, I always try to ask: why? From memory, I've basically gotten these answers:
sudo apt-get install rustc cargo
, cloned your repository and then tried to run cargo build
." In effect, they are trying to port workflows that they expect to work (probably from other language ecosystems) to Rust, and then get miffed that compilation breaks. Sometimes this even translates into errant bug reports. (Although, I think that's largely solved by Cargo's rust-version
feature, once I get around to using it.) I also think people interpret this as a signal that Rust is "too unstable," in the sense that too many people are using language features that require a very new compiler. My experience is that most C or C++ libraries don't do this, and tend to update their language version (e.g., C++11 to C++14) pretty conservatively. So even folks using a conservative Linux distro will be able to do the apt-get install foo
and then build random projects using foo
. But for Rust, that doesn't really work. Instead, they really have to go use rustup
, which is just going to be foreign to lots of folks not familiar with the Rust ecosystem.I think (1) is basically some combination of an awareness/workflow/perception problem. I think there's probably something to (2), but I just don't know enough concrete details about any specific use case to really know for sure. And in particular, I suspect (2) is kind of also a perception problem. Both (1) and (2) seem somewhat rooted in the idea that Rust and its ecosystem move so much faster than the things it tends to supplant that it tends to be really noticeable.
Now I do think perception problems are real problems, but I'm not convinced they reflect concrete use cases that argue strongly for a conservative MSRV policy. (By "conservative," I'm thinking that there is roughly support for N-9
releases, i.e., one year.)
Note that this is a policy proposal for libc, not for other crates maintained by the libs team. After we get consensus on an MSRV policy for libc, we can consider what policy we want for other crates.
OK, so I think this is really where my real concern with this change is. libc
is basically as foundational as you can get as a library. Deciding an MSRV policy for it comes very close to effectively deciding an MSRV for the broader ecosystem. IMO, even if the MSRV is really just a perception problem, that's a pretty big deal. (I say "very close," because as @thomcc mentioned, if people hate this enough, they'll starting pinning libc
and then that's going to be a cure that is worse than the disease. So technically, users of libc
can still enforce their MSRV by either pinning libc
in libraries (very bad) or pinning libc
in their Cargo.lock
(which is totally fine). But other than that, libc
will end up deciding the MSRV for a lot of folks I think.)
Since libc
is foundational and because I suspect this policy is probably more aggressive than extant MSRV policies in the ecosystem (being a little hand wavy there), I do think that we should try to broadcast this policy question before making a final decision on it. That is, I would like to make a good faith effort to give folks an opportunity to provide their opinion on this because of the impact this will have on the broader ecosystem. Basically, I want to avoid a scenario where this happens:
N-4
. So that gets merged and shipped out.libc
has almost nearly forced them to make their policy more aggressive than what they had end up asking what's going on in the libc
tracker.Maybe I'm overplaying it, but that's my sense. So I'll register a concern for that:
@rfcbot concern opportunity for ecosystem to give feedback
Otherwise, for me personally, I don't think I feel strongly for or against this change. I've long wanted good reasons for a conservative MSRV policy, and I've had a hard time finding them. Maybe this issue will force out some good ones that I've not heard of before. But I can definitely empathize with the annoyance of having to wait so long to use new Rust stuff, so I get where this is coming from.
Note that this is a policy proposal for libc, not for other crates maintained by the libs team. After we get consensus on an MSRV policy for libc, we can consider what policy we want for other crates.
OK, so I think this is really where my real concern with this change is.
libc
is basically as foundational as you can get as a library. Deciding an MSRV policy for it comes very close to effectively deciding an MSRV for the broader ecosystem.
I was attempting to narrow the question to libc, not because I think it likely that other official crates can or should realistically be more conservative than libc, but rather, I wanted to separate the questions so that if someone wanted to argue "we should support X" they didn't have to simultaneously argue whether that meant X for libc or for everything.
If we do agree on N-2 for libc, then our realistic choices of policies for other official crates are N-2, N-1, or N. I'd tend to suggest N-2 for "foundational" crates like libc or libz-sys.
- "My company pins its Rust version in CI and it takes a lot of resources to upgrade."
I more or less meant this scenario. In my case it was more like "several teams need to build the code in several infras, and asking others to update their infra too often is a nuisance because they have other more important things to do, so in practice the required Rust toolchain gets updated once in 0.5-1.5 years".
Crate versions still can be updated more often because it doesn't require similar coordination. Also few crates change MSRV that often in practice so most of the dependencies still build in a year and don't need to be pinned to prevent updates. So the N-3,4 policy actually makes libc move much faster than majority of other less fundamental crates.
So the N-3,4 policy actually makes libc move much faster than majority of other less fundamental crates.
Not necessarily, but only possibly:
Only upgrade if there's a feature we actually need.
It just means that libc could bump the MSRV if needed. As you said...
few crates change MSRV that often in practice
@petrochenkov Can you say more please? Here are the things I'm personally interested in:
@shepmaster
So the N-3,4 policy actually makes libc move much faster than majority of other less fundamental crates.
Not necessarily, but only possibly:
Only upgrade if there's a feature we actually need.
It just means that libc could bump the MSRV if needed. As you said...
few crates change MSRV that often in practice
I think you're technically correct, but I think the color you're adding here might give the wrong impression. There's a burden to supporting older MSRVs, and while @joshtriplett's post does include the language "Only upgrade if there's a feature we actually need," I think the word "need" there is doing a lot of work. For example, "need" might mean, "we want to use a new std API and don't feel like writing version sniffing code in build.rs
to do conditional compilation" or it might mean "we have so much version sniffing already that adding more is just not worth the burden" or it might mean "we want to use a new Cargo.toml
feature and there's no easy way of introducing such things conditionally." Frankly, all three are somewhat reasonable interpretations, but it's not totally clear which one will be followed in practice to me. So it might worth clarifying what exactly is meant by "need" here.
For example, if "need" is interpreted broadly, then I think it's very likely that libc
will end up bumping MSRV more rapidly than existing "core" crates. But if "need" is interpreted narrowly, then it might actually be pretty rare for libc
to bump MSRV.
Someone else I'd like to hear from is @dtolnay. As I understand it, you maintain a Rust 1.13 MSRV for serde. This is despite requiring at least some non-trivial effort to do so. I don't think I've seen you opine on why exactly you do this. Could you share your perspective here?
(Also, I have not actually broadcast this issue to a wider audience yet. Are people okay if I do that? Or do folks think we should wait on that for a bit or go about it a different way? As a former mod, I always feel a little weird shining spotlights on GitHub issue threads, since it can go sideways pretty easily.)
@BurntSushi
Why do you think it's justified for the (probably the entire) Rust project to spend effort maintaining a conservative MSRV because it is costly for y'all to upgrade?
Because it's not that much effort in practice. It's "some extra maintenance in libc alone" vs "some extra maintenance for all the people with slower Rust toolchain update schedules", the choice seems pretty clear to me.
If upgrading Rust is really that costly, can you say more about why using older versions of crates isn't an acceptable alternative path for you?
It's acceptable, and the issue is not a showstopper, but newer crate versions are not purely cosmetic and do fix relevant bugs sometimes (or add relevant Linux constants or whatever, in case of libc).
Because it's not that much effort in practice.
Eh. The libc
build.rs
is pretty gnarly, and all of those cases get turned into conditional compilation knobs that get sprinkled throughout the code. I know I'm pretty averse to doing that (I've done it before on several occasions), so I end up just usually waiting to use a new feature in cases where I've tried to maintain a "conservative" MSRV.
It's doubly annoying because finding real tangible use cases for conservative MSRV has proven difficult. So it just feels like a lot of hand wringing for... not much in exchange.
It's "some extra maintenance in libc alone"
Certainly not! It's extra maintenance for everyone who tries to maintain a conservative MSRV. If libc
got more aggressive about it, it might spur everyone else to get more aggressive too, thus reducing maintenance burden for a whole bunch of folks. Whether that's bigger or smaller than the "people with slower Rust toolchain update schedules" is actually totally unclear to me. I have no idea how widespread "slow toolchain update schedules" is in practice. I'm sure it's not just you (I've heard of others complaining about it), but I truly do not know the scale of it.
I think you're technically correct, but I think the color you're adding here might give the wrong impression.
I was not attempting to add color, or even opinions, just surface the wording concerns which you have now expanded upon.
For SNAFU, I maintain a Rust 1.34 MSRV, although I don't use a build.rs for newer versions. Instead, I use feature flags for each version that I need and use default features to enable a "new enough" version of Rust. This places the burden on people who want to use SNAFU with an "old enough" version of Rust, while still allowing a wide range of Rust versions to use SNAFU.
This works for me because I can write the majority of my code targeting old Rust versions. This does mean I'm locked to edition 2018, among other things, but that isn't amazingly painful for me in practice.
I have received feedback from Linux distro users that are using old versions of Rust so I know that there are real people out there affected by this.
Is there a list of what "new Rust" features the libc crate would like to make use of?
- "I did
sudo apt-get install rustc cargo
, cloned your repository and then tried to runcargo build
." In effect, they are trying to port workflows that they expect to work (probably from other language ecosystems) to Rust, and then get miffed that compilation breaks. Sometimes this even translates into errant bug reports. (Although, I think that's largely solved by Cargo'srust-version
feature, once I get around to using it.) I also think people interpret this as a signal that Rust is "too unstable," in the sense that too many people are using language features that require a very new compiler. My experience is that most C or C++ libraries don't do this, and tend to update their language version (e.g., C++11 to C++14) pretty conservatively. So even folks using a conservative Linux distro will be able to do theapt-get install foo
and then build random projects usingfoo
. But for Rust, that doesn't really work. Instead, they really have to go userustup
, which is just going to be foreign to lots of folks not familiar with the Rust ecosystem.
I totally understand that folks in the Rust ecosystem generally don't want to support traditional distro toolchain timelines, especially in enterprise where that can stretch 10 years or more. However, I do want distro Rust toolchains to be a viable option, and for RHEL we've tried to make that happen by keeping a more aggressive "rolling stream". That doesn't mean we ship every 6-week Rust release, but we do update the Rust toolchain in each RHEL minor release, every 6 months. (I also maintain Fedora's Rust, and we do ship every update there on all active releases.)
Now, with our own internal development pipeline, the version we put out in a RHEL minor release will already be 3-4 months old when it comes out. That's our latest version for the next 6 months, so it will be 9-10 months old by the time the next one comes around. For example, in May 2022, RHEL 8.6 and 9.0 shipped Rust 1.58, and those will be in use until 8.7 and 9.1 around November.
People also don't upgrade RHEL releases right away. Conservatively, padding that out to a year for an MSRV update policy is probably good enough, and if they still haven't updated they'll have some impetus. And FWIW, we did choose a one-year MSRV policy for rayon
(although it's currently sitting quite a bit older).
I know many people won't care about RHEL, but we're trying to find a good compromise there.
2. "My company pins its Rust version in CI and it takes a lot of resources to upgrade."
I more or less meant this scenario. In my case it was more like "several teams need to build the code in several infras, and asking others to update their infra too often is a nuisance because they have other more important things to do, so in practice the required Rust toolchain gets updated once in 0.5-1.5 years".
And in some cases, this sort of case is also "outsourced" to the distro update process.
As someone who delivers software written in Rust to RHEL environments, I would really like to hear from users who are developing on RHEL using the distro rustc
/cargo
instead of rustup
. We jump through hoops to deliberately avoid using distro packages when possible, on account of their age.
I don't have hard customer data to offer here, but I think there's also a chicken-and-egg problem, that Rust won't even be a viable option for those customers at all if the ecosystem won't support this kind of use.
I mean for our usage, the rest of RHEL packages are an obstacle, so we never even considered using a Rust toolchain from the distro. So I wonder if this is a more severe chicken-egg scenario.
Fair enough. :)
I'm one of the maintainers of pyca/cryptography, a popular Python package (top 20 downloads from PyPI most days) with a risk component. I can provide some limited stats on versions of Rust we see.
Our current MSRV is 1.41, and our next release intends to increase our MSRV to 1.48.
Looking at downloads of our source artifacts (i.e., users who need to compile themselves, excluding pre-compiled artifacts) we have rustc version data for about 1.6% of those downloads (because tracking this information requires a relatively recent pip).
Here is the % of such downloads where the rustc version is >= to each target, from the past 30 days:
1.41.0: 99.6%
1.42.0: 97.2%
1.43.0: 97.2%
1.44.0: 97.2%
1.45.0: 95.9%
1.46.0: 95.9%
1.47.0: 95.9%
1.48.0: 95.7%
1.49.0: 91.3%
1.50.0: 90.3%
1.51.0: 90.3%
1.52.0: 90.2%
1.53.0: 89.7%
1.54.0: 89.4%
1.55.0: 89.2%
1.56.0: 89.2%
1.57.0: 87.4%
1.58.0: 86.6%
1.59.0: 75.7%
1.60.0: 48.9%
1.61.0: 40.3%
1.62.0: 20.5%
Hopefully this presentation of the data makes sense.
If/once cargo can use the rust-version
from Cargo.toml
to automatically pick the latest version that works on the compiler that's used, this could go a lot smoother. Then if libc 0.2.135 requires Rust 1.70, cargo will just pick libc 0.2.134 for users on Rust 1.69, such that nothing breaks that didn't need the latest libc.
Two scenarios for "My company pins its Rust version in CI and it takes a lot of resources to upgrade." I've seen personally:
The problem with "just using older versions of crates" is that sometimes you want to upgrade something for unrelated reason. Like, you use "latest-async-framework-my-company-fork-5.0.0.beta.8", and you want to switch to "latest-async-framework-5.0.0" b/c that has much fewer bugs, and, while doing that work, you discover you need to upgrade rustc as well.
For both of these scenarios as I've encountered them, N-1
would create measurable pain, N-4
would be OK, and N-8
would be very comfortably OK.
A minor thing is build caching. When using a shared Rust compilation cache, it's nice to be able to stay on a specific toolchain for longer, because then you get fewer rebuilds. This is maybe a less important reason for libc, though, as upgrading libc means recompiling much of the world anyway.
(And as a sidenote: it's naive to say that this policy is just for libc. libc's msrv policy basically is the baseline msrv policy for the entire ecosystem... at least for msrv at maximal dependency versions.)
It's definitely worth polling major ecosystem crates for what their msrv policy is and for what users they hold that policy. E.g. tokio has committed to a 6 month msrv through 2024 at a minimum.
Plus it's worth restating: a rust-version aware version resolution will make msrv bumping much smoother.
(And as a sidenote: it's naive to say that this policy is just for libc. libc's msrv policy basically is the baseline msrv policy for the entire ecosystem... at least for msrv at maximal dependency versions.)
Yeah I think @joshtriplett and I discussed this briefly above. I think that's the idea behind considering the MSRV policy for libc
first. It starts with the most expansive question.
@alex Thank you, that's useful information. Would it be possible to get, rather than the last 30 days, three chunks of this information? Specifically: 2022-04-07 to 2022-05-19 (while 1.60 was newest), 2022-05-20 to 2022-06-30 (while 1.61 was newest), and 2022-07-01 to today (while 1.62 was newest)?
It would help to see how much correlation that has. (Which in turn will depend on how many of your users get rust from rustup vs a distribution package; I would guess many do, but this will at least give us a conservative bound to consider.)
Also, what does
relatively recent pip
mean here? How old is that version of pip? (We'd like to take into account the possible bias towards recent versions there.)
Only upgrade if there's a feature we actually need.
One downside to an as-need basis is if you upgrade too infrequently, your users might build up an expectation that diverges from what is documented. clap has always had an N-2 policy but went so long without updating the MSRV that we got backlash when updating our MSRV to something less than a year old.
If/once cargo can use the rust-version from Cargo.toml to automatically pick the latest version that works on the compiler that's used, this could go a lot smoother.
One effect here is that we might weaken ecosystem-wide "live at HEAD" property. Today, the world mostly looks like this:
Cargo.lock
after cargo upgrade
No idea what's the size of the effect here.
after
cargo upgrade
I'd personally expect cargo-upgrade to ignore rust-version, as it's already making semver-incompatible upgrades which you'll need to fix.
cargo-update / cargo-generate-lockfile are the ones which I'd hope to be rust-version aware.
(This was likely just a typo as to which tool was intended to be written.)
lib-rs has data on estimated supported msrv (not policy, but what crates actually build on): https://lib.rs/stats#rustc
Notable are the big bumps at 1.31 and 1.56, for the new editions, and the gap is even more pronounced for crates getting updates. This also shows that at the very least, a 3 year (one edition-length) policy ought to be enough for anyone. From those charts I'd wager anything beyond stable-12 (1½ years) is excessive.
cc @kornelski: is it possible to get a version of that chart weighted by rdeps/lib-rs-rank/similar? (Added as a feature request.)
Personally, I think I'd be fine with a libc MSRV bump just so stuff like unions and more idiomatic Rust can be used, instead of having to deal with poor imitations that break things like strict provenance.
As such a foundational crate, libc should be as conservative as possible here, and maybe that means discussing MSRV on an as-needed basis instead of something tied to N or N-4.
Regarding use-cases, I do think the distro-build one is going to be underrepresented in discussions here. I have several (closed-source) applications that people expect to just be be able to compile directly on their distro, where things like security and audting concerns make rustup nonviable.
Regarding distros, which distro? Ubuntu 16.04 is out of "standard support" but it packages Rust 1.47.0 and LTS extends to 2026. Is libc going to support 1.47.0 as an MSRV for the next 6 years?
Separately, I wonder about the dissonance of using a years-old toolchain to compile very recent packages. I feel like there is something else other than a conservative MSRV that we should use to help people who are in a bind for "security and auditing concerns".
I also worry about not hearing from distro users. I would really like to know what kind of people are using Rust 1.41 to compile cryptography
and where they are even getting it.
@saethlin Debian stretch is almost certainly where the 1.41 come from.
@joshtriplett Sure, happy to pull those. The functionality has been in pip since 21.2 which came out 2021-07-24, so almost exactly a year ago. I do think its reasonable to assume that older pip likely also means older rustc, but that's an assumption.
For '2022-04-07' to '2022-05-19':
1.41.0: 99.7%
1.42.0: 98.0%
1.43.0: 98.0%
1.44.0: 98.0%
1.45.0: 97.2%
1.46.0: 97.2%
1.47.0: 97.2%
1.48.0: 96.7%
1.49.0: 93.8%
1.50.0: 93.0%
1.51.0: 93.0%
1.52.0: 93.0%
1.53.0: 92.2%
1.54.0: 91.9%
1.55.0: 88.5%
1.56.0: 85.7%
1.57.0: 80.7%
1.58.0: 58.1%
1.59.0: 46.2%
1.60.0: 33.7%
1.61.0: 0.2%
1.62.0: 0.1%
For '2022-05-20' to '2022-06-30':
1.41.0: 99.3%
1.42.0: 97.4%
1.43.0: 97.4%
1.44.0: 97.4%
1.45.0: 96.6%
1.46.0: 96.6%
1.47.0: 96.6%
1.48.0: 96.3%
1.49.0: 92.1%
1.50.0: 91.2%
1.51.0: 91.2%
1.52.0: 91.2%
1.53.0: 90.7%
1.54.0: 90.6%
1.55.0: 90.5%
1.56.0: 90.5%
1.57.0: 88.1%
1.58.0: 73.4%
1.59.0: 60.5%
1.60.0: 44.8%
1.61.0: 28.3%
1.62.0: 0.0%
For '2022-07-01' to '2022-08-01':
1.41.0: 99.9%
1.42.0: 97.0%
1.43.0: 97.0%
1.44.0: 97.0%
1.45.0: 95.5%
1.46.0: 95.5%
1.47.0: 95.5%
1.48.0: 95.2%
1.49.0: 91.2%
1.50.0: 90.2%
1.51.0: 90.2%
1.52.0: 90.1%
1.53.0: 89.6%
1.54.0: 89.3%
1.55.0: 89.1%
1.56.0: 89.0%
1.57.0: 87.2%
1.58.0: 86.5%
1.59.0: 77.0%
1.60.0: 52.8%
1.61.0: 44.0%
1.62.0: 31.4%
using Rust 1.41 to compile
py/cryptography
Well, given the volume of complaints from py/cryptography adding a build dependency on Rust breaking docker builds — I'd wager a not insignificant amount of "users" building py/cryptography are in fact absolute-minimum-required docker containers which pull from the distro.
I realize that this is kind of explicitly out of scope, but... MSRV is purely "how long do we give downstream to upgrade their toolchain" and no more while rustc keeps the explicit policy that only the latest stable toolchain is supported.
@alex Thanks! Pulling those together into one table, and trimming the beginning:
After 1.60 | After 1.61 | After 1.62 | |
---|---|---|---|
1.56 | 85.7% | 90.5% | 89.0% |
1.57 | 80.7% | 88.1% | 87.2% |
1.58 | 58.1% | 73.4% | 86.5% |
1.59 | 46.2% | 60.5% | 77.0% |
1.60 | 33.7% | 44.8% | 52.8% |
1.61 | 28.3% | 44.0% | |
1.62 | 31.4% |
Or, relative to each release:
After 1.60 | After 1.61 | After 1.62 | |
---|---|---|---|
N-4 | 85.7% | 88.1% | 86.5% |
N-3 | 80.7% | 73.4% | 77.0% |
N-2 | 58.1% | 60.5% | 52.8% |
N-1 | 46.2% | 44.8% | 44.0% |
N-0 | 33.7% | 28.3% | 31.4% |
I do want to acknowledge the bias inherent in data collected from a no-more-than-year-old version of pip
, which does mean we're not going to count users who got pip (and possibly also rustc) from a multi-year-old stable distribution.
Nonetheless, this data still seems useful, and it seems worth at least considering an N-3 or N-4 policy, rather than N-2. At the very least, it's something we should consider when evaluating the tradeoffs of expending additional maintenance burden on support for older versions.
I also want to call attention to a factor that is likely to contribute to making this a more...vigorous discussion than the norm.
Whatever policy we choose here is likely to have an effect on the rest of the Rust crate ecosystem, both because many crates indirectly depend on libc, and because many crate maintainers are likely to treat this as guidance or as something they can defer to when justifying their own MSRV policies.
And, whatever policy we choose here is unlikely to get changed in the future, as we aren't likely to want to go through this more than once.
Finally, this is something where not having a policy is worse than most policies we could choose, and the status quo of uncertainty (defaulting to "don't change anything, support everything") is painful and will only grow more so with time.
As a result, this is the perfect storm of wide impact and now-or-never sentiment that combine to produce a highly charged atmosphere and what often become very long and exhausting threads.
I would like to raise that issue now, in the hopes that it might be, if not averted, at least lessened.
Please keep this in mind when posting in this thread:
One potential consequence of a more-aggressive MSRV policy which I've not seen mentioned yet (apologies if I've missed it) is the creation of one or more libc
"backports" derivatives, which could potentially fracture part of the ecosystem. If someone routinely needs changes made to a newer version of libc
but with an older Rust compiler, one option would be to fork libc
, declare an older MSRV, and then start cherry-picking commits (or adding back the conditional compilation steps which are proposed to be removed). The result could be some crates depending on e.g. libc-msrv-1.47
rather than libc
, which seems less-than-ideal for a foundational ecosystem component.
OTOH, the construction and adoption of such a derivative crate would be a good indication that there is a use case for an older MSRV, and I wager changing libc
's MSRV to support additional versions of Rust in the future would be less of a hot-button topic than the present one.
libc-msrv-1.47
crates existing would actually be better than a situation where people pin libc versions, though.
I'm pretty worried about the possibility of pinning since that's a way I've seen crates resort to time and time again to address MSRV issues, despite it being an approach with very serious drawbacks.
Does cargo tell crates-io what version it is when talking to the site? This would be an area where we could benefit from a little bit of telemetry to know what versions people are using to download packages. This'd give a direct estimate of the versions people are then using to build packages.
With that data, it'd be simple to say to e.g. support an MSRV such that 90% of (distinct?) connections to crates-io can compile the crate. The crates-io package mostly cares about users compiling the crate from crates-io; it's not entirely infeasible to ask distros which want to stick to an older rustc to downpatch the package or gate functionality themselves.
Unfortunately saying we should start collecting that information if we don't already is not going to help, and collecting telemetry, however anonymized, is always a questionable privacy trade-off.
Here's data from running cargo check on all crates:
It looks like cargo's default edition has the biggest influence on de-facto ecosystem wide msrv.
edition has the biggest influence on de-facto ecosystem wide msrv.
Which suggests a 3 year compatibility window. Released Rust 2021 -> switched libc MSRV to the first Rust 2018 rustc release (and perhaps bumped the crate's edition to 2018 too). Rleased Rust 2024 -> switched libc MSRV to the first Rust 2021 rustc release, etc.
I suspect most of those are not really considered MSRVs, so I'm unsure they make sense to take into account. I think data from crates.io would be much more interesting. Either way, I think a 3 year compatibility window is far too conservative, and is trading a high maintenance cost for a very small number of users. 6 months (N-5) or at most 1 year (N-9) seem like much better targets.
6 months seems good since it seems to have proven to be at least somewhat successful in tokio, since that's the compatibility window it uses.
To elaborate on my previous comment, since it seems easy to misinterpret: My comments around pinning are just I just think we should have a plan to mitigate the potential fallout it may cause. Possibly through education that pinning causes significant non-obvious issues (or perhaps even a warning if people attempt to publish libraries with pinned crates... probably not though -- there are totally reasonable situations when you should pin dependencies, such as proc macro re-exports and such).
There is some risk that this causes a libc-msrv-whatever
crate to pop up which has a lower MSRV, but honestly, I'm not that worried. Even if that happens, it's not like libc type compatibility across the ecosystem is that important -- the only libc types that I frequently see in APIs are primitive integers, which are (hopefully) going to compatible anyway.
Regarding reasons for using older MSRV, I'm linking my comment from the other dicsussion: https://github.com/rust-lang/libc/pull/2845#issuecomment-1186961010
TL;DR: upgrading a crate requires acceptable auditing but getting trustworthy Rust binary is too annoying. This is also a counterargument to people potentially implying that refusing to upgrade Rust but bumping crate version is a cognitive dissonance.
One more note:
people interpret this as a signal that Rust is "too unstable,"
Indeed, I'm a super-big fan of Rust and don't perceive the language as unstable but definitely do perceive the ecosystem as unstable. This feeling got stronger lately.
6 months seems good since it seems to have proven to be at least somewhat successful in tokio, since that's the compatibility window it uses.
"tokio uses this compatibility window" plus "tokio is popular" does not mean "this compatibility window is a necessary or contributing factor for popularity".
Does cargo tell crates-io what version it is when talking to the site? This would be an area where we could benefit from a little bit of telemetry to know what versions people are using to download packages. This'd give a direct estimate of the versions people are then using to build packages.
Unfortunately, that would count every build separately, rather than counting unique users; as a result, it would disproportionately count CI systems, and disproportionately count CI that runs more often. I don't think that would give reliable data.
Unfortunately, [crates-io data] would count every build separately, rather than counting unique users
Thus my hedge for "distinct" connections (e.g. by IP), but I do agree that the data is likely too noisy to be directly useful.
For Tokio, our policy is:
A side effect of the way we maintain our LTS branch is it makes things easier to pin that branch to its MSRV for CI to pass.
Some of our users like pinning to the LTS releases to have more stability (Tokio behavior doesn't accidentally change from under them) while also being able to get security patches. They tend to update their Tokio as rarely as possible.
All this to say, that we would prefer our transitive dependencies to have fairly stable policies w.r.t MSRV. Of course, I also acknowledge that it adds a maintenance burden, so in the end, y'all feel free to do what works best for you. I don't think it would be terribly hard for us (Tokio) to maintain a fork of libc and maintain our own MSRV requirements.
I don't think it would be terribly hard for us (Tokio) to maintain a fork of libc and maintain our own MSRV requirements.
Surely we can come up with a better solution than "either maintain a long MSRV compatibility window or we'll fork".
I don't think it would be terribly hard for us (Tokio) to maintain a fork of libc and maintain our own MSRV requirements.
Surely we can come up with a better solution than "either maintain a long MSRV compatibility window or we'll fork".
I mostly meant this to emphasize I'm not coming in saying "Tokio needs X, you must do it". I know maintaining old versions is added work.
In a previous libs team meeting, we discussed MSRV policy for the libc crate, and we debated between an N-1 policy and a slightly more conservative policy such as N-3 or N-4. We ended the meeting without having settled on a specific consensus, though it seemed like we were roughly on the same page.
Rather than make ad-hoc changes based on a non-specific MSRV policy, I think we should go ahead and nail down an MSRV policy for present and future changes.
I'd like to propose the following concrete policy:
Note that this is a policy proposal for libc, not for other crates maintained by the libs team. After we get consensus on an MSRV policy for libc, we can consider what policy we want for other crates. Nonetheless, libc MSRV policy seems likely to serve as a defacto lower-bound for supported Rust versions among other libs team crates, and among many crates in the ecosystem.
For further information on the benefits of this: the PR dropping support and version detection for older versions of rustc was able to remove ~1600 lines of code, and that's not counting subsequent work that may be able to drop some of the wrapper macros. This represents a substantial improvement to maintainability, and may make it easier in the future to move to more automation to keep libc updated with the latest platform APIs.