Closed thejpster closed 1 year ago
It's an interesting idea, certainly. However it wouldn't fit directly with rustup
's current model. That's not to say it couldn't be made to work, just that it'd be awkward.
If you're insistent on doing it via a Travis matrix then it'd have to be rustup
or else travis would have to have some special behaviour to achieve that. Currently rustup
is able to just go and ask static.rust-lang.org for channel data. I wonder if, perhaps, it might be sensible to have a rolling set of stable-{1,2,3,4,5} channel symlinks on there?
At that point, we could add support for the requisite channel name shape to rustup
and wouldn't need special handling otherwise.
I guess the Debian equivalent would be old-stable and old-old-stable, if the numeric component causes an issue.
I think oldstable and oldoldstable are more plausible just because -
is used to separate the elements of toolchain names. In theory ~
could be used too but that introduces a whole host of usability issues potentially. @pietroalbini If I were to propose adding some channel symlinks (or however we deal with stable/nightly vs. the numbered/dated equivalents currently) which approach do you think would fly?
Do we know from channel data the stable version? We could do the arithmetic easily enough in rustup to synthesise new pseudo channels, no?
Such synthesis would change the download-channel-install-channel model which IMO would be moderately awkward; though as I said before yes it'd be possible. However the ideal would be if the pseudochannels were handled on static.rlo, otherwise we have to start faffing with working out which patch level is the newest for a version (e.g. if stable is 1.35.0 and the user wants stable~1 do we install 1.34.0 or the more useful 1.34.2 ?) So many roundtrips would make rustup update
expensive and slow. And yes, I know we do that for nightly backtracking but that's only one dimension (date) rather than more (channel / minor / patch)
I think a good approach would be to create new manifests for 1.MINOR.x
(like 1.43.x
, 1.34.x
...) that always point to the latest patch release of that minor version. This would allow to:
stable~N
in just two requests (as we can subtract from the minor of the current stable, and fetch the .x
channel of the resulting version).1.31
you'd use 1.31.x
as the MSRV toolchain).Yeah, I like using stable~<number>
too. Feels like using git reset HEAD~<number>
.
if stable is 1.35.0 and the user wants stable~1
The good default to me is using latest patch release. As patch releases often backport soundness and compatible/build fixes.
Edit: minor -> patch
if stable is 1.35.0 and the user wants stable~1
The good default to me is using latest minor release. As minor releases often backport soundness and compatible/build fixes.
I'd call that the patch release, as the 35
in 1.35.0
is the minor, and the 1
is the major. But yes, I can't see why anyone would want to not pick up the highest patch release for a given major/minor pair.
So I like both @pietroalbini 's idea of MAJOR.MINOR.x
being the latest PATCH
for MAJOR.MINOR
though I'd note that might confuse people who do numeric ordering and assume the dots separate numbers. and I like the idea of stable~1
being "one back from current stable". The .x
links have the advantage that they're only updated on a release of that MAJOR.MINOR
series, but the stable~n
would have to be updated every stable release. If we do the latter, I don't think we should do it for beta or nightly though.
I'd prefer not to have stable~N
as actual manifests on static.rlo (keeping them up to date is going to be a bit of a pain). Could we go with MAJOR.MINOR.x
on static.rlo, and implement stable~N
on the rustup client?
I guess we could have a go. I don't like the double-downloading of manifests but I suppose if the user is opting in then they're prepared to pay the cost. So we define stable~N
to mean major(STABLE).(minor(STABLE)-N).x
and then fetch that? Will we backfill static.rlo with the MAJOR.MINOR.x
for all the releases? Presumably that'd be fairly cheap.
Yeah pretty sure we can backfill static.rlo.
We now have major.minor channel files present on the dist server, so we're a step closer to being able to do this.
If someone wanted to have a go at implementing the channel parsing and handling the double-manifest-download then we can see how it works.
@rbtcollins This seems a bit complex, and I don't have a concrete timeline, but I still want to give this one a try. Could you please mark this issue as claimed?
Before actually trying to implement this change, I'll try to share my thoughts on this issue below:
It seems that the main use cases for this stable-DELTA
syntax is in CIs, so at which level should we de-sugar it?
IIRC, in today's model, stable
, 1.71
and 1.71.0
are 3 separate toolchains (assuming the current stable is on 1.71.0
):
stable
will become 1.72.0
weeks later.1.71
will stay on 1.71
but still upgradable (e.g. from 1.71.0
to 1.71.1
).1.71.0
will get pinned to 1.71.0
forever.To me, it will be too weird to actually make stable-DELTA
an upgradable toolchain just like stable
, so I'm considering to define the semantics of stable-2
to be exactly the same as 1.69
.
That is, I don't expect anyone to actually perform a +0.1
upgrade on those.
Does this sound acceptable? (cc @kinnison @joshtriplett)
The current syntax for a toolchain spec is:
CHANNEL[-YYYY-MM-DD][-TARGET]
... where:
CHANNEL
is one of stable
, beta
, nightly
, X.YYY
or X.YYY.ZZ
.TARGET
can be partial, like i386
, pc-windows
, msvc
, i386-pc-windows
, i386-msvc
or pc-windows-msvc
instead of the complete form i386-pc-windows-msvc
.... and I suggest to add a possibility for the 2nd part: an unsigned integer of at most 3 digits, so that it becomes:
CHANNEL[-YYYY-MM-DD|-WWW][-TARGET]
This would then accept the following (still assuming the current stable is on 1.71.0
):
stable-2
=> 1.69
stable-31-pc-windows
=> 1.40-pc-windows
stable-999
[^sem]beta-2
[^sem]nightly-2
[^sem]1.23-4
[^sem]1.23.0-4
[^sem][^sem]: Allowed by the syntax, but might cause semantic errors.
How I would direct rustup/cargo to do a build with the resolved toolchain? Will cargo +stable-2 build
work?
How I would direct rustup/cargo to do a build with the resolved toolchain? Will
cargo +stable-2 build
work?
@thejpster It is still a bit too early to talk about this. As I see it, there are two ways of defining the semantics of stable-2
:
Making it a shorthand of 1.69
(if stable
is on 1.71
), which won't receive any +0.1
updates.
If this happens, then with stable-2
you have just installed 1.69
and nothing more, since stable-2
is not actually a channel in this case. The actual de-sugaring happens every time the syntax is used, so every time you are using something like cargo +stable-2 build
, you are sure that you are actually building with the current stable-2
(but that also means it will instead require 1.70
if the stable
version has bumped).
Making it a "virtual channel" installed as separate toolchain.
If this happens, then stable-2
behaves just like stable
(but 2 versions behind the current stable
): they are distinguished from major.minor
toolchains, will receive +0.1
updates, can get out of date (and if that happens you need to update them with rustup update
), and finally, cargo +stable-2 build
requires a stable-2
toolchain installation to work.
I'm still hesitant about this, initially I wanted to go with 1. (as posted above), but after some discussions with other devs it seems to me that 2. is a better option. After all, these two semantics should be equivalent on CIs, but what if someone wants to do this on their own dev machine?
But on the other hand, I don't currently know if this virtual channel thing does exist. If it doesn't, I might have to invent the wheel myself :o
There are other similar things to decide, for example:
stable-2
play with Rust 2.0
? Maybe stable-0.2
?-
is minus
instead of a connector? stable~2
and stable^2
also seem dangerous... Some suggested something like 2-before-stable
or stable-minus-2
for clarity [^1].[^1]: I was suddenly reminded of my time spent on SICP... What about -2+stable
? 😂
I think I agree that (2) would provide a better UX, although it would be more work to implement.
(I also agree the minus
vs connector thing needs to be thought about.)
Thank you!
IIRC Debian use oldstable and oldoldstable.
Thank you!
IIRC Debian use oldstable and oldoldstable.
@thejpster Thanks for the help!
This is indeed very clear without using any weird characters! I'll consider using this name or at least some variant of it :)
I think there will also be use cases for old-old-old-old or even old-old-old-old-old-old-old-old, so I'd probably try to get a number in there rather than relying on repetition like this.
There are two design points referenced in the recent message.
1) what name should rustup accept to specify one of these stable + offset things, 2) are they real channels or virtual.
1) I think we should accept stable-${decimal}
: debians versioning schemes are very rigorous but also influenced and limited by compatibility with early versions of dpkg and apt, and we are not constrained by them.
2) per https://github.com/rust-lang/rustup/issues/2291#issuecomment-738317030 there are now manifest files for major.minor releases which get updated. A bit of history: we used to have channels like stable and exact versions but not major.minor. This meant you couldn't install rust 1.72 via metadata - rustup had to decide you meant 1.72.0 and get you that instead. This means that we already have actual real channels for each concrete floating version that might be referenced in an MSRV if we can name them.
The proposal linked above is that when rustup encounters a name with a version offset (@kinnison used ~
, I think we should use -
as ~
has different semantics in Debian and it would be confusing to use that), rustup will examine the stable
version number, subtract the version offset from the minor
part, discard the point
part, and then use that channel as the toolchain.
The question about whether we end up with e.g. a toolchain on disk called stable-2
is a good one. I propose we don't do that and take the option (1) described in https://github.com/rust-lang/rustup/issues/2291#issuecomment-1690899882
What do you think of this possible bit of documentation for the feature.
# MSRV 'last-N releases support'
Rustup can support MSRV for projects that want to build up to N releases back from the current stable release by building using the syntax `stable-N`. For instance `rustup toolchain install stable-3`. This will:
- install the stable toolchain if not installed (this provides the anchor point by which `stable-3` is calculated
- install a toolchain `MAJOR.MINOR` with the version number determined by subtracting 3 from the stable version numbers minor part and dropping the point part
When building for MSRV support, use a [toolchain override](...url skipped) of `stable-N` to build against the current major.minor toolchain as described above. If your stable toolchain is out of date, this may build with an older major.minor than you might expect. Point releases on the major.minor toolchain are only applied when a `rustup toolchain update` is performed - the same as for any other toolchain.
Reasons why I propose this:
- I think we should accept
stable-${decimal}
: debians versioning schemes are very rigorous but also influenced and limited by compatibility with early versions of dpkg and apt, and we are not constrained by them.
I'm still wondering if there will be a 2.0 moment of Rust, and how we might design this feature to be future-proof in that respect.
- reduces the number of distinct toolchains people need: if they have 1.69, and 1.69 is needed for a stable-N scenario, their 1.69 is reused. We have open bugs about preventing additional downloads of toolchains, so it makes sense to me to take that into consideration in new feature development.
One downside of this approach, I imagine, would be too many old toolchains on one's disk if they choose to install locally. 1.69, and 1.70 after that... There doesn't seem to be a "garbage collection" mechanism.
I'm still wondering if there will be a 2.0 moment of Rust...
When there is Rust 2.0, I believe rustup is very likely to ship breaking changes along with that. Thus not an issue from my pov.
I wonder if we can support this without mandating that the stable toolchain is installed? Or do we think that ~everyone has stable installed anyway so there's not much point?
@djc I think @rbtcollins wants stable to be installed as an anchor point so that we can effectively know what stable-2
actually means (so that stable-2
is always 2 versions behind the installed stable
rather than the real stable
).
This IMO is also different from my original proposal, in which I wished to resolve stable
on the fly and calculate the actual version based on that, but I think it is somewhat more consistent.
Ideally we can do both based on whether stable
is installed, but I'm afraid that will cause more confusion than necessary...
I realise I didn't cover one of the design constraints. Because rustup proxies are in the critical path for build time, we need a design that doesn't pay (much) overhead, and that works when the internet is down.
The (much) overhead - its probably ok to run rustc, or parse the stable manifest or both, when a proxy is invoked with no RUSTUP_TOOLCHAIN set. Once set, it needs to be set to a concrete value that won't trigger resolution again.
I'm not sure what I'm proposing is necessarily the best, but it seems ok at current thinking to me.
RUSTUP_TOOLCHAIN=stable-2 cargo build
, the rustup proxy does the translation to get e.g. 1.69, resets RUSTUP_TOOLCHAIN for the child process, and away we go.stable
to a new version but doesn't attempt a build with stable-2
, when the internet would be needed again.cargo +stable-2 build
won't download anything.Using stable-2
as a virtual-channel would have similar properties with the following differences:
Something I've adopted in my projects and am also starting to use in cargo is to have RenovateBot automatically update my MSRV.
e.g. see https://github.com/rust-lang/cargo/pull/12381
Would something like that existing lower the priority of implementing a feature like this directly in rustup?
An alternative possible design is that rustup should expose +msrv
which could read the rust-version
from the project's Cargo.toml
and run that. Maybe simpler and more directly solves the use case? I suppose this would be roughly analogous to using the rust-toolchain file.
An alternative possible design is that rustup should expose
+msrv
which could read therust-version
from the project'sCargo.toml
and run that. Maybe simpler and more directly solves the use case? I suppose this would be roughly analogous to using the rust-toolchain file.
@djc Yes, that looks interesting, which could even free us from caring about stable
.
The tricky part is how this works in workspaces, which don't natively have a rust-version
. They could have a workspace.package.rust-version
, which could alleviate the problem, but the interactions with --manifest-path
might be surprising? I guess cargo +rust-version --manifest-path=foo/Cargo.toml
could not pick up the rust-version
from foo/Cargo.toml
-- but then we already have the same problem today with rust-toolchain
files?
Forgive me if I quote myself:
Currently we set an MSRV like a stake in the sand, and then time moves on and the stake gets further and further away. One of our crates has an MSRV of 1.30 - thats 72 weeks old at the time of writing. What I'd like to do is bring that MSRV along with us - trailing it behind us on a piece of string.
@thejpster I'm not sure exactly what you are trying to convey. Is it that you want the +stable-X to be trailing along without you having to update the rust-version
? If so, don't you want your Cargo manifest metadata to be correct as the MSRV increases over time?
My OP I think dates before rust-version existed so yes, we wanted MSRV to move without changing anything in the repo.
So how/when are you going to the rust-version
in your crate's Cargo.toml
? Or are you going to publish your crate without rust-version
even though there is some limited MSRV support?
I don't know that anyone has thought about it. I would probably publish without a rust-version and say in the README "If you're more than three versions behind stable, this might not work"
If the answer is "don't set a rust-version", then this feels too specialized of a feature.
The ideal we should be working towards is setting rust-version
cargo upgrade
(and the part that will get merged into cargo) will prevent upgrades across MSRVThen I'm happy to declare this 3 year old ticket as OBE and close it.
IIRC @epage has used Renovate to send automated PRs to advance the rust-version
? Might be a good option.
Yah - rustup toolchain files are not a good fit for MSRV management; that belongs with cargo.
Describe the problem you are trying to solve
In Rust-Embedded land, we are struggling with the management of our Minimum Supported Rust Version. That is, the promise we make to our users that compilers all the way back to our MSRV will compile this code, and we won't use any features stabilised after that compiler was released.
Currently we set an MSRV like a stake in the sand, and then time moves on and the stake gets further and further away. One of our crates has an MSRV of 1.30 - thats 72 weeks old at the time of writing. What I'd like to do is bring that MSRV along with us - trailing it behind us on a piece of string.
Describe the solution you'd like
Basically in my CI files I want to "rustup install stable-2", and if stable is 1.40, it would install 1.38. If stable was 1.51 it would install 1.49. Then I can set my Travis CI matrix to build on:
And I never have to touch the CI file again.
Notes
At the moment, to achieve a rolling MSRV we would have to touch the CI file every six weeks when there's a new stable release. That or
rustup install stable
, grok the current version with some grep/sed foo, subtract two minor versions (not easy in Bash) and then install that version.