rust-lang / libs-team

The home of the library team
Apache License 2.0
115 stars 18 forks source link

MSRV policy for libc crate #72

Open joshtriplett opened 2 years ago

joshtriplett commented 2 years ago

In a previous libs team meeting, we discussed MSRV policy for the libc crate, and we debated between an N-1 policy and a slightly more conservative policy such as N-3 or N-4. We ended the meeting without having settled on a specific consensus, though it seemed like we were roughly on the same page.

Rather than make ad-hoc changes based on a non-specific MSRV policy, I think we should go ahead and nail down an MSRV policy for present and future changes.

I'd like to propose the following concrete policy:

Note that this is a policy proposal for libc, not for other crates maintained by the libs team. After we get consensus on an MSRV policy for libc, we can consider what policy we want for other crates. Nonetheless, libc MSRV policy seems likely to serve as a defacto lower-bound for supported Rust versions among other libs team crates, and among many crates in the ecosystem.

For further information on the benefits of this: the PR dropping support and version detection for older versions of rustc was able to remove ~1600 lines of code, and that's not counting subsequent work that may be able to drop some of the wrapper macros. This represents a substantial improvement to maintainability, and may make it easier in the future to move to more automation to keep libc updated with the latest platform APIs.

thomcc commented 2 years ago

"tokio uses this compatibility window" plus "tokio is popular" does not mean "this compatibility window is a necessary or contributing factor for popularity".

This is not what I meant. I mainly meant that I don't hear people complaining about it often (the opposite is true -- anecdotally people seem to find it to be a good policy). That said, I wasn't quite familiar with all the details of how tokio's policy operates (in practice it seems to hold MSRV longer than I had believed, and perhaps longer than we'd want).

shepmaster commented 2 years ago

Even if that happens, it's not like libc type compatibility across the ecosystem is that important

I was under the impression the opposite was true. Like, I feel like it’s been said that there will never be a new semver-incompatible release of libc because fracturing the ecosystem (again?) would be too traumatic. Am I just making up things here?

CAD97 commented 2 years ago

We definitely would prefer not to provoke another "libcpocalypse". From a cursory search, though, it looks like the scale of the libcpocalypse was down to crates using libc = "0" and not just from including libc in public APIs?

CAD97 commented 2 years ago

(Separate note)

I want to draw sufficient attention to clarifying tokio's policy: while the Tokio MSRV policy is 6 months, combining this with a 6 month LTS of that tokio release means that Tokio is actively releasing guaranteed-in-support patches targeting up to a year-old compiler, and extended-best-effort support to year-and-a-half (or perhaps even two years, if the LTS promotion happens not to a fresh minor release).

Of course, within the LTS releases Tokio can more easily consider just pinning a maximum version of out-of-tree dependencies to ensure maximal-versions dependency resolution succeeds on their LTS MSRV.

I continue to hold my position that both

[^1]: assuming that the dependency does not break semver compatibility and then refuse to unbreak it

and the combination of those axioms is that MSRV is only meaningful with rust-version-aware or minimal version resolution.

carllerche commented 2 years ago

If/once cargo can use the rust-version from Cargo.toml to automatically pick the latest version that works on the compiler that's used, this could go a lot smoother. Then if libc 0.2.135 requires Rust 1.70, cargo will just pick libc 0.2.134 for users on Rust 1.69, such that nothing breaks that didn't need the latest libc.

i think you are correct that this would make most of the questions moot. The only remaining question I can think of is how to manage security patches. What I have found with teams that don't update their dependencies is that, when there is a security patch to pull in, they end up hitting issues w/ upgrading their dependencies which can delay resolving the security issue.

udoprog commented 2 years ago

6 months seems good since it seems to have proven to be at least somewhat successful in tokio, since that's the compatibility window it uses.

"tokio uses this compatibility window" plus "tokio is popular" does not mean "this compatibility window is a necessary or contributing factor for popularity".

To me the default standard barring some other reason could be to adopt a conservative MSRV encapsulating that of what a reasonable quorum of popular downstream crates are doing by policy. Largely in an attempt to follow the quorum decision of the downstream community. Otherwise libc would be making an active decision to force downstream crates to change their policies (or fork) which seems like the greater harm over the alternative.

So far the other reasons I've picked up is that the existing state of the libc crate is hard to maintain (rust-lang/libc#2845). And applying the policy used by Tokio (6 months or older) would allow for adopting up to Rust 1.58. Doing that would allow the maintainers of libc to remove all the version detection proposed in rust-lang/libc#2845 which seems to solve the maintenance reason for changing the MSRV policy in the libc crate. Since the scope of this proposal is to solve the question for libc, no further issues nor reasons currently seem to be enumerated here that motivates picking a less conservative MSRV policy.

And if a future good reason comes up to adopt a less conservative MSRV the decision can always be revisited when it has been enumerated.

joshtriplett commented 2 years ago

@shepmaster

I was under the impression the opposite was true. Like, I feel like it’s been said that there will never be a new semver-incompatible release of libc because fracturing the ecosystem (again?) would be too traumatic.

The primary issue with libc major version differences would be the c_* types. However, we can solve those in two different ways: have the older version re-export the newer, or have them reuse the types now available in core::ffi. Both of those, however, would result in a dependency on newer Rust.

Fortunately, all of that would only apply if we bumped libc to a semver-incompatible version, which I don't believe we should on the basis of upgrading MSRV.

joshtriplett commented 2 years ago

@udoprog

what a reasonable quorum of popular downstream crates are doing by policy

Many crates are not maintaining support for old MSRVs out of policy. Many crates are maintaining support for old MSRVs because one or more of 1) they're exhausted dealing with complaints, 2) their dependencies are asking, without necessarily having an end user in mind, or 3) they're supporting hypotheticals that again don't have an attached end-user.

I absolutely think that there are people in the Rust community who would like support for very old versions of Rust. I also think there are fewer such people than is commonly believed among crates maintaining old MSRV support.

While in some cases changing the MSRV of libc might push a downstream crate to change its MSRV policy, in other cases changing the MSRV of libc may free a downstream crate to change its MSRV policy.

So, this is one reason why I'd like to make sure that we decide our MSRV policy on the basis of a substantial number of actual end users, not on the basis of downstream crate policies that are a proxy for nonspecific end users. (To be clear: crate policies chosen because of specific end-user needs are much more important here than policies set for other reasons.)

And if a future good reason comes up to adopt a less conservative MSRV the decision can always be revisited when it has been enumerated.

Once this policy has been set, I doubt we'll have the energy to want to revisit it in the future.

m-ou-se commented 2 years ago

If/once cargo can use the rust-version from Cargo.toml to automatically pick the latest version that works on the compiler that's used, this could go a lot smoother. Then if libc 0.2.135 requires Rust 1.70, cargo will just pick libc 0.2.134 for users on Rust 1.69, such that nothing breaks that didn't need the latest libc.

i think you are correct that this would make most of the questions moot. The only remaining question I can think of is how to manage security patches. What I have found with teams that don't update their dependencies is that, when there is a security patch to pull in, they end up hitting issues w/ upgrading their dependencies which can delay resolving the security issue.

Users of older-than-latest-stable Rust are already missing out on security patches to Rustc and std. We generally don't release new minor releases of older Rust versions when a security issue is found.

udoprog commented 2 years ago

@m-ou-se

Users of older-than-latest-stable Rust are already missing out on security patches to Rustc and std. We generally don't release new minor releases of older Rust versions when a security issue is found.

Maintainers of distributions such as Debian are committed to backport such patches per their own policy. Preferably with the help of upstreams, but even without.

thomcc commented 2 years ago

I think it's been mentioned, but I don't believe we always make an announcement about issues like this that get fixed.

Perhaps debian is watching the set of issues labeled either P-critical or P-high and perhaps something like regression-from-stable-to-stable, but this is not something I've witnessed (admittedly this doesn't say that much), nor is even that guaranteed to be complete the complete story (it certainly has some amount of noise as well, as P-high/P-critical may cover a number of non-security issues as well).

I'm not sure that's a particularly great situation to be in, but it's the one we're in, and the fact kind of is that... users who care about security patches probably need to be on the latest stable version.

In an effort to make this as easy as possible, we try very hard not to break code when updating, so there should be very little reason not to do so.

udoprog commented 2 years ago

@joshtriplett

Many crates are not maintaining support for old MSRVs out of policy. Many crates are maintaining support for old MSRVs because one or more of 1) they're exhausted dealing with complaints, 2) their dependencies are asking, without necessarily having an end user in mind, or 3) they're supporting hypotheticals that again don't have an attached end-user.

The end-user standard that you're proposing would require feedback between end users and crates that simply does not exist. I do in fact believe that if such evidence is systematically collected and presented to the community it would be seriously considered as a reason to change policy naturally. But it isn't, so that argument can't be properly assessed quantitatively.

As for 1.), which exhausted maintainers are being referred to here?

That leaves the following reason (emphasis mine):

While in some cases changing the MSRV of libc might push a downstream crate to change its MSRV policy, in other cases changing the MSRV of libc may free a downstream crate to change its MSRV policy.

This is certainly true by definition, but that in itself is not a reason. A reason would be some negative that occurs because a crate is required to maintain an overly strict MSRV policy. Such as the maintenance woes of libc.

Once this policy has been set, I doubt we'll have the energy to want to revisit it in the future.

I think we should appreciate the large impact that policy decisions in libc have in practice, something several people have emphasized above as well. And as a result move slowly because the only way to garner real feedback on this change is to perform it and see who comes running*.

Frankly there's an even more important question here: Why should it be within the libs-team's purview to strongarm an MSRV policy for the rest of the ecosystem rather than first attempting something softer like consensus-building?

Unless I'm missing something, the reasons seem to be:

And then there seems to be a simple belief that end-users should dictate MSRV policy - a group we currently can't garner feedback from - rather than something like the maintainers of widely used crates.

* Some more broader thoughts on setting policy

Since we lack the necessary feedback to judge policy right now, the only way we can judge it is by breaking the policy and observe which downstreams come back complaining. It is probably more common that those downstreams end up being other crates before end-users, because those that take MSRV seriously tests their requirement so even a break in policy in libc shouldn't have time to propagate to end users (unless something goes very wrong).

joshtriplett commented 2 years ago

Nobody is proposing "strongarming" anyone. The libs team and the folks maintaining libc are evaluating the amount of time required to maintain it and the tradeoffs of doing so in various ways (including how doing so limits other potential maintenance improvements), and are attempting to evaluate the downstream effects of potential policy changes.

And then there seems to be a simple belief that end-users should dictate MSRV policy - a group we currently can't garner feedback from - rather than something like the maintainers of widely used crates.

Crates with such policies are (or at least should be) attempting to serve end users, directly or indirectly. The point I was making is that "this is our policy" carries less weight than "this is what our users tell us they need", which in turn carries less weight than "this is what our users actually use".

Kixunil commented 2 years ago

@joshtriplett appreciate your attempt to rigorously understand the situation. Unfortunately it seems even more complex. By counting current end users we're excluding people who avoid Rust because "it's too unstable" and who would reconsider their position if a move in the opposite direction happened: if the whole ecosystem attempted to maintain conservative MSRV. Sadly, I don't see a way to figure out how significant this group is.

I also think crates having a policy should have slightly higher weight than what you seem to be expressing. There's very a few, if any, crates where the crate author doesn't use the crate (IOW "for fun but serious crates). If a crate has a MSRV policy it means at least one of these is true:

I don't remember ever coming across anyone who picked their policy arbitrarily. It's actually the other way around: crate has no policy, gets asked to have a specific policy then the author decides to accept or reject.

the8472 commented 2 years ago

Maybe I missed it, but I haven't seen an argument why putting an older libc version into a project's Cargo.lock is not sufficient for those who use older rustc versions. It's not like the old libcs will magically stop working.

joshtriplett commented 2 years ago

@Kixunil The most common reason I've seen for a crate to maintain an MSRV policy is because they maintainer of one of their dependencies asked them to. And I've chased several of those dependency chains recursively, and found them to end in "I don't know"s regarding who actually needs such a policy.

m-ou-se commented 2 years ago

@m-ou-se

Users of older-than-latest-stable Rust are already missing out on security patches to Rustc and std. We generally don't release new minor releases of older Rust versions when a security issue is found.

Maintainers of distributions such as Debian are committed to backport such patches per their own policy. Preferably with the help of upstreams, but even without.

They maintain their own old copy of Rust, but they still want to use the latest version of libc from crates.io? I was under the impression they also maintain similarly old copies of the Rust software (including dependencies) that they intend to compile with their version of Rustc.

epage commented 2 years ago

As for 1.), which exhausted maintainers are being referred to here?

I unofficially lowered clap's MSRV because of pressure that I didn't want to deal with for the sake of library example code to use clap where the library transitively follows tokio's MSRV.

alex commented 2 years ago

@m-ou-se Distributions users want to use a new libc crate with the old rustc they got from their distro. We have the same pattern in the Python ecosystem.

m-ou-se commented 2 years ago

Those users should be using the latest stable Rustc. I was under the impression that the outdated Rustc version that some distributions ship is only included for compiling the Rust software included with those distributions. (Which usually includes an (old) copy of the libc crate, etc.)

Darksonn commented 2 years ago

I use the compilers shipped by my distro for all programming languages I use, except for Rust. Doing so is a quite common workflow.

BurntSushi commented 2 years ago

@Kixunil I've tried chasing things down too. That's what I started my initial comment with. So far, it looks like all MSRV use cases have fallen into those two buckets. (Which are "I need to use some pre-approved Rust compiler from my distro" and "upgrading Rust is painful/costly for $reasons.")

I think both of those use cases are very real and they can be valuable to support. But what I'm still not clear on is just how widespread they are. I do typically maintain a somewhat conservative (albeit not Rust 1.13 levels of conservative) MSRV in "popular" crates that I maintain. Why do I do that? It is definitely not for personal reasons. And it isn't also because someone explicitly asked for it for that crate. My reasons are:

Bottom line: I personally don't care about MSRV and I would be happy to not care about it at all. However, given my own pace of development and the kind of code I write, it is usually pretty cheap for me to maintain a mildly conservative MSRV. (Think 6-12 months.) So in that context, I think, "if it doesn't cost me much and probably helps out some folks in some practical scenarios, then that seems like a win." To be honest, that's kind of what I think libc should do too, absent really compelling data to the contrary.

It seems like the difficulty here is in picking the time window. N-4 is on the conservative end of what has been proposed, which I think roughly corresponds to 24 weeks, which is just shy of 0.5 years. N-5 would get us to 30 weeks, which is firmly over 0.5 years. N-5 does not feel like an undue burden to me, and it is pretty close to the existing proposal and it seems like it would satisfy at least some folks here (even if we don't all necessarily agree with the reasons for why a particular MSRV is useful). AIUI, for example, this would permit removing all of libc's current version sniffing.

I think the case for an MSRV more conservative than N-5 is difficult to make. @joshtriplett, I know you mentioned that you think we won't want to revisit this, but I personally accept the point made by others that it might be difficult to truly know just who is impacted by this without actually doing it. So one path we could take here is to provisionally accept an N-5 MSRV policy, but leave the door open to backing it out and making it more conservative if compelling use cases arise. It does draw this discussion out, but it seems difficult to truly make a firm decision that we won't revisit up front here.

udoprog commented 2 years ago

@joshtriplett

Nobody is proposing "strongarming" anyone. The libs team and the folks maintaining libc are evaluating the amount of time required to maintain it and the tradeoffs of doing so in various ways (including how doing so limits other potential maintenance improvements), and are attempting to evaluate the downstream effects of potential policy changes.

That is fair but I'm using it in a descriptive manner: an enacted policy in libc that either forces Tokio to change its MSRV or fork libc to maintain theirs would simply be an act of "strongarming" to me (not to mention other crates I don't know about). No one's seemingly demonstrated an outweighing harm to the libc project of adopting an MSRV policy compatible with Tokio's.

As long as the obvious harm of forcing a really popular downstream to change policy is not on the table (without good reasons!) I have no issues with exploring the larger much more complex problem space. I'd also like to emphasize that I personally don't care about MSRV support - that is not how I use Rust. I only care that Tokio has an established policy and it doesn't matter if it has no benefit to the users of the project because I can't prove it one way or another.

matklad commented 2 years ago

If libc changes MSRV to, say, N-1 tokio can still maintain MSRV of N-4. To do so, it will need to:

As long as libc doesn't bump major just to bump MSRV, downstream crates can maintain more conservative MSRV policy with non-trivial, but also not prohibitive, amount of work.

carllerche commented 2 years ago

Users of older-than-latest-stable Rust are already missing out on security patches to Rustc and std. We generally don't release new minor releases of older Rust versions when a security issue is found.

This is true, but it isn't all or nothing. Also, I know there are large users of Rust willing to backport security patches themselves if needed (though, I'm sure they would rather have an official LTS policy).

udoprog commented 2 years ago

@matklad Keep in mind that the effect of that proposal is that all teaching materials and tutorials that a user which ends up with an outside the officially supported MSRV window compiler might be exposed to would have to teach that. And not just for libc but every crate that doesn't abide by its MSRV.

I think the workable solution has been mentioned above a few times: that cargo uses rust-version as an additional <=VERSION constraint when solving dependencies.

taiki-e commented 2 years ago

@matklad

  • ensure that tokio's CI for rustc N-4 uses Cargo.lock with pinned version of libc or some other equivalent of -Zminimal-versions

Note that this will not work if the upstream is not compatible with -Zminimal-versions. Also, -Zminimal-versions itself has several pitfalls, and AFAIK very few projects check this in the right way on CI. (See cargo-minimal-versions' readme and https://github.com/tokio-rs/tracing/pull/2015#issuecomment-1092561025 for details)

(FWI, https://internals.rust-lang.org/t/msrv-specific-code/17045/7 has another discussion on using -Zminimal-versions for MSRV.)

the8472 commented 2 years ago

Keep in mind that the effect of that proposal is that all teaching materials and tutorials that a user which ends up with an outside the officially supported MSRV window compiler might be exposed to would have to teach that.

That seems like quite an overstatement of the impact. Not everything has to be covered in every tutorial, especially niche features such as ancient distros. If cargo warns about a version mismatch and the warning suggests ways to solve the problem that may already be sufficient for the users to resolve them on their own.

udoprog commented 2 years ago

Keep in mind that the effect of that proposal is that all teaching materials and tutorials that a user which ends up with an outside the officially supported MSRV window compiler might be exposed to would have to teach that.

That seems like quite an overstatement of the impact. Not everything has to be covered in every tutorial, especially niche features such as ancient distros. If cargo warns about a version mismatch and the warning suggests ways to solve the problem that may already be sufficient for the users to resolve them on their own.

Unless rust-version is specified, cargo won't know about the version mismatch until a crate fails to compile (e.g. item foo requires unstable feature X). I do support helpful diagnostics that leads you to solve the problem, but I'm not sure how this one is supposed to be implemented in practice. And when it is implemented it would still take time to "boil down" to the slow moving places where Rust is vendored.

Once rust-version becomes available the solution I and others have touched on is also available which in my mind supersedes the need to solve it in other ways. But it also suffers the problem that a new version of cargo has to boil down so it's optimistically at least a year down the road (unless it's already been implemented?) before it can improve user experiences.

Kixunil commented 2 years ago

@joshtriplett interesting! I do think that crates having MSRV for no reason is probably not great but even "I want people to perceive Rust as reasonably stable" is a good reason. I do agree with others wanting cargo to resolve MSRV, would help me a lot too. A more problematic case is when crates bump major version because of API change and then crates depending on them have to pick version in Cargo.toml. It's often impossible to satisfy both and then downstream consumers may end up with duplicate dependencies. :(

@BurntSushi thanks for thoughtful explanation! I do have very similar experience regarding maintenance burden. I can usually comfortably maintain 1.48 or even 1.41 MSRV but sometimes it's a pain - things depending on new tokio features are biggest offenders (e.g. prost) Unfortunately N-5 is not nearly enough to support Debian stable but looks like I'm in minority and I'll respect that.

seanmonstar commented 2 years ago

Going down a different branch, this part of the proposed policy is pretty squishy:

Only upgrade if there's a feature we actually need.

For instance, if I look at the libc PR that spawned this issue, I can subjectively say that many of the newer upgrades are not "needed". Is removing ~10 lines a huge maintenance improvement? Is that what is meant by "actually needed"? I would have assumed such a sentence would mean something like MaybeUninit or async was introduced in the compiler, thus allowing for UB to be fixed, or for a fundamentally new API.

It seems this sentence could benefit from coming up with criteria that would allow people to identify when a feature is truly "needed".

the8472 commented 2 years ago

I understood that point as saying that the MSRV is merely an upper bound, even-lower versions may happen to work if nobody had a reason to bump it. I.e. it won't necessarily be bumped like clockwork.

mitsuhiko commented 2 years ago

I’m largely with @BurntSushi here. I try to maintain my crates very conservatively and typically it’s dependence of mine that make me move up. In fact I noticed almost a month after the latest edition release that a lot of the ecosystem moved up at once were before it was trivial to maintain compatibility down to 1.41 or 1.48.

One big reason for me to support much older Rust versions is that it’s just more convenient as a maintainer to me. I am not likely to be on the receiving side of the user frustration of being the culprit of moving up the minimum version and I also don’t typically benefit of newer features much as a library author.

As to why users don’t upgrade most answers have already been given but there is another one worth mentioning. Sometimes you find yourself with a specific compiler version and replacing it can be more work than beneficial in that situation. For instance I routinely come across dated Rust compilers on build images. Yes it’s not a lot of work to upgrade in the general situation but it’s still an extra burden that can cause unnecessary frustration for a few minutes to an hour depending on the complexity of the setup.

I have been plenty of times in situations where I was complaining as a user why a certain dependency all the sudden interrupts me to upgrade a compiler when I was trying to fight a completely different fire.

BurntSushi commented 2 years ago

@seanmonstar Yeah I talked about "need" above (although it's a long discussion now): https://github.com/rust-lang/libs-team/issues/72#issuecomment-1187625438

I do think we should try to change "need" to be a little clearer. I do personally favor a more expansive view... Otherwise if it is taken literally, then it seems pretty likely to me that a build.rs is just going to continue building up complexity until there "happens" to be something that can't be solved by conditional compilation.

CAD97 commented 2 years ago

Initial suggestion: an MSRV bump is considered "needed" if it impacts the public API surface of the crate. If it just represents internal cleanup (e.g. const _ and const-panic for static assertions) it is not considered "needed". This is a generous baseline definition.

The strictest possible definition of "needed" is that using the new functionality changes the API surface in a nonadditive way... but I don't think libc would ever run into that, as I'm pretty sure that also implies the use is breaking.

Due to libc's nature as mostly just public type definitions, this (looser) definition would make most beneficial MSRV bumps considered "needed." However, a definition between the two is certainly subject to bikeshed about where the line is drawn.

A completely different definition of "needed" would be that taking advantage of the feature in the implementation is nonadditive (e.g. again replacing static assertion hacks with const-panic). However, I'd personally recommend against taking a "needed" policy which does not show up in the public API, as that is much more difficult to justify to downstream.

What I would recommend is defining "needed" in as close to objective manner as possible, and then aggressively upgrading rust-version/MSRV as soon as the "needed" bump passes the time-based requirement. Delaying MSRV bumps outside of policy only leads to expectations of the defacto MSRV support.

joshtriplett commented 2 years ago

@seanmonstar This is precisely why we need to nail down a policy, so that we don't have to justify each individual change. When I wrote "need" in this context, I meant only that we don't systematically bump rust-version forward on every stable release, only when we actually use a feature from newer Rust. I'm not suggesting that we should hesitate or need to justify using a feature from a Rust version that fits within our MSRV policy.

There's a trade-off in that aspect of the policy as well. If we don't systematically bump MSRV, and we go a while without requiring newer Rust, people may come to expect the older MSRV. But hopefully, having a written policy will address that sufficiently.

seanmonstar commented 2 years ago

While I personally wouldn't select a policy that lenient, this is also why I brought it up: the language "actually need" is too subjective, and will require you to justify yourself. If that's the policy the libs team wants, the language would be better written as "We upgrade when there is a new feature we can use."

joshtriplett commented 2 years ago

I've updated the language in the top post to hopefully make that unambiguous.

seanmonstar commented 2 years ago

I'm not suggesting that we should hesitate or need to justify using a feature from a Rust version that fits within our MSRV policy.

The new language up top:

Only bump MSRV when we use a feature requiring newer Rust.

Putting on my opinionated hat, my preference is that consideration would be taken to analyze "how much does keeping these few lines cost the maintainers" vs the "how much does upgrading cost the users". As a library maintainer, I've very rarely felt that new features in rustc were useful enough to require the newer version. Certainly, people who give away free code don't owe anyone anything. But it feels user-centric to consider that a cost to users is usually orders of magnitude higher than a cost to maintainers.

Another option that is always available to a team of maintainers, if they feel their time is limited, is to bring on more maintainers.

Kixunil commented 2 years ago

Regarding "needed" one thing that came up in several crates is cosmetic changes - matches! macro, skipping arguments in println!()&co, various one-line functions that added to core e.g. unsigned_abs...

It'd be nice if "needed" avoided these but I'm not sure how to rigorously define it.

joshtriplett commented 2 years ago

The interesting question is not just "when can we do cleanups", it's "a useful new feature just came out that we could really use, when can we use it". Our policy needs to be written to account for that case, in which we are likely to merge a change using it the moment it meets the time window. If a long time passes without any such feature, that does not mean that the MSRV policy has changed.

Once we've decided what window of Rust versions we want to support, there's no added value in making that window longer on the basis of "that feature isn't very important", because downstream users can't count on that wider window in the future. Given that, once we've decided what window of Rust versions we want to support, I don't think there's value in (and in fact I think it's a substantial negative to) justifying each proposed change that would bump MSRV.

joshtriplett commented 2 years ago

Also, this is not a matter of "cost to users vs cost to maintainers"; this is two different costs to users. A longer MSRV has real consequences to users. Among other things:

Also, in case it seems like most of the desired features are in the past, sometimes the distant past, here are some future features libc may want to use as soon as they're available:

(That's just off the top of my head; I'm sure there are others.)

eminence commented 2 years ago

As a library author, I have attempted to maintain a fairly conservative MSRV (these days it's 1.48), under the thinking that "all things equal, more conservative is more better"). My MSRV-bumping policy is poorly defined, and amount to basically "convince me that a MSRV bump is worth it".

However, I found myself quite annoyed that I wasn't able to seriously to deploy my own MSRV policy because in practice my MSRV policy ends up being beholden to the MSRV policy of my dependencies (several of which had no documented policy and ended up changing the MSRV inadvertently). Asking upstream consumers to pin specific dependencies is possible (mentioned earlier in this thread), but not a great experience.

So I am following this issue quite closely because I strongly believe that the MSRV policy for libc will become a de-facto standard (and I am quite interested in having some standard that I can rely on and adopt myself)

workingjubilee commented 2 years ago

Hello!

I believe rust-lang/libc should not be asked to support a version older than Rust 1.31 (roughly analogous to "when the Rust 2018, so, prior Edition released"), because the needs to support older versions can be accomplished by appropriate version pinning and yes, backporting and forks if necessary. This would be More Than Enough for Debian LTS stable support, and would be between an N-25 and N-50 support policy.

That said, I also have to put forward this: I believe the libc crate, itself, may be ill-formed. By using one crate to represent many platforms, many of which only vaguely respect libc's idea of a POSIX and/or ISO C interface, it ties many quite-distinct platforms to a single interface and risks calling updates to any of them a "breaking change" for all of them. This has raised many questions with, for instance:

Linux (glibc):

Linux (musl):

BSDs:

joshtriplett commented 2 years ago

As a hypothetical alternative policy, to gauge potential interest and seek alternate solutions:

I'm not normally a fan of treating MSRV bumps as a semver matter. However, as a potential compromise solution, would it be easier for folks with downstream expectations about MSRV if the policy was:

This would have downsides, among them that people would encounter more duplicate copies of crates, and that we would need to take care to avoid ecosystem skew. However, if this would potentially make it lower friction for us to adopt new rust features, it's worth considering as an alternate policy.

Kixunil commented 2 years ago

@joshtriplett if you mean issuing breaking release when MSRV is bumped it'd make things significantly worse. I have a very similar situation with prost and it's super-ugly. Rust-version-aware dependency resolution is really the only way forward for things like this. Maybe this discussion is motivating enough to implement it?

BurntSushi commented 2 years ago

Yeah I'm very opposed to bumping semver for an MSRV bump.

joshtriplett commented 2 years ago

@BurntSushi I am as well, I'm just less opposed to that than I am to (for instance) waiting 6-12 months before being able to use new Rust features.

workingjubilee commented 2 years ago

I threw out the "we should at least not ask for more than 1 Edition behind" partly because I think we should at least be putting forward some lower bounds for what we're willing to consider, but also because I think the phenomenon people have observed of "suddenly, a bunch of people roll forward at once" is somewhat informative? A value like N-4 is probably setting the wrong cadence.

Lokathor commented 2 years ago

Debian stable seems to be 1.48 (S-14), oldstable and oldoldstable go as far back as 1.34 (S-28) on some arches, but are 1.41 (S-21) on the more common arches. Certainly "1.31 because of the 2018 edition" is a completely reasonable bound on how far back we go. That's S-31, that's a very long time in the rust world.

"Stable-X, or a semver break if we want to go newer sooner" sounds good to me because I like the idea of people learning to adapt to a small semver break here and there, but I suspect most of the rest of folks would call that a completely wild thing to invite into the ecosystem.

I suspect that, in the long run, picking a larger value of X to be more compatible with linux distribution would be useful. I suspect that having any rolling MSRV policy will unstick most of the problem with the libc crate. I suspect that once the MSRV is rolling regularly that alone will be enough to take the pressure off. Even if the delay is larger than the proposed 2 or 4 (which I think are way too short), the fact that it's only a delay, not a "the libc crate is stuck on 1.13 forever!" should be enough to ease the maintenance tension by quite a bit.

According to the one source of any version usage we had, 85-90% of users were within S-4, which means that 10-15% of people get cut off by a window as short as S-4. Should we be okay with that for such a foundational crate? I don't think so. For a crate like libc, extra compatibility should be a great virtue. It's just gotten a little out of hand by going all the way back to 1.13 is all. Even S-10 should be fine as long as it's a rolling minimum.