Updating rust-version going forward

[leighmcculloch]

I think we need to discuss the plan for the minimum supported rust version of crates, and our testing plan around rust versions, now that Protocol 20 is released.

Context

Rust binaries are built with whatever version of Rust is installed in most cases. Rust products are typically pinned to a "channel" of Rust, such as "stable" rather than a specific version. But a rust version can be pinned for building by specifying it in rust-toolchain.toml or by simply installing only that version of Rust.

Rust libraries are built with whatever version of Rust is installed, and libraries can specify a minimum supported rust version (MSRV). If a library requires v1.75 and v1.74 is installed, an error occurs blocking builds. There is no concept of a maximum supported rust version.

Rust releases rarely involve backporting to past stable releases. Bugs get fixed in the next stable release.

What we're doing now

The status quo is that we track the latest released rust version immediately upon its release. By tracked, I mean we use the latest version in our CI, and we require at least the version we tested with at release. We require the latest version by specifying it in the rust-version field of all of the crates' Cargo.toml files which is snapshotted when the crate is published to crates.io.

The existing plan has some advantages:

• Everyone developing on the crates is building with the same version of rust across the entire stack.
• Everyone is building locally with the same version we use in CI.
• Everyone can use the latest in tooling which is constantly being improved (e.g. rust-analyzer).
• If important bugs/security-bugs/features are released in a new version of Rust, it's easy to benefit from them with a small diff.
• New users are likely to pick the latest version of Rust, not the version of Rust we tested libs when at release, and so by upgrading immediately we are also testing our stuff asap with what others are using.

The existing plan wasn't heavily discussed, but it was motivated by the benefits above.

The existing plan isn't perfect:

• There's no guarantee that the version of rust people use with the Rust SDK will be the same version we tested with. We released all the crates having tested them with Rust v1.74, but folks are likely already using the Rust SDK and Env with Rust v1.75. But per the benefit above, because we update immediately we start running our tests on new versions quickly and if we detected a test breakage we could address that in a patch release.
• If we released a minor/patch of stellar-core, we might need to update the Rust version it uses if a lib was updated already, which is updating a dependency in a minor/patch release. Arguably one of the most tested dependencies in our dep graph though.
• If we released a minor/patch of Rust SDK and had updated the Rust version, folks might need to update their Rust version. Some organizations won't tolerate updating toolchains as quickly.

Alternative for moving forward

Set MSRV of crates to v1.74, run CI with v1.74, and pin stellar-core to v1.74 to minimize change.

Core Ubuntu releases would be using a build of Env that is as consistent as can be with past builds. Source and Homebrew tap builds of Core may not use v1.74. We won't be doing any testing of core with newer versions of Rust with existing Core CI.

Contract and app developers who have started developing won't need to update their rust-version when they update minor/patch versions of the Rust SDK, Env, Stellar XDR lib, etc. We'd probably need to test the lib releases with both v1.74 and the latest release to catch breakages but those additional CI runs would be parallel.

If a meaningful bug or security-bug is released, we may want to re-release with newer versions of Rust, and bump the MSRV if it's particularly meaningful to core functionality of the libs.

Changes required:

• Remove the update rust-version job from each repo
• Change CI in all Rust repos to build and test with both the MSRV and latest stable
• Change any product binaries to utilize rust-toolchain.toml to pin to a version they decide to use (prob v1.74 for stellar-core)

Thoughts

I think we should start doing the alternative. I think down the road there will be folks that are disrupted by us doing MSRV updates in minor/patch releases, particularly larger organizations that have rules around what toolchains they use. It also hands control over what version to release with to products (e.g. stellar-core) and libraries get used in a range of versions anyway. Devs can also still use newer versions of Rust and tooling locally on the Rust libs.

Whichever way we go I'd like to document what we're doing once we make a decision.

cc @graydon @MonsieurNicolas @anupsdf

[leighmcculloch]

I've implemented the suggested alternative flow for the stellar-xdr lib:

• https://github.com/stellar/rs-stellar-xdr/pull/341

I was motivated to implement it because that repo uses clippy in CI and with the new rust just released clippy is failing. So it was an easy two problems one solution to pin CI to run clippy on the msrv for now.

This ☝🏻 can serve as a example and reference to what we'd need to do elsewhere if we decided to change what we're doing.

[graydon]

I concur with your suggestion here. Fwiw stellar-core already pins to channel = "1.74" in rust-toolchain.toml.

[leighmcculloch]

With @MonsieurNicolas's comment in slack and @graydon's comment, I'm going to move forward with this approach then.

Applications will need to make their own decision, they can release with any version of Rust they want that's greater than the env/sdk's MSRV:

• [x] stellar-core – already pinned to 1.74 per @graydon's comment above. I think it should pin 1.74.1 because that was the exact version used for the last release. https://github.com/stellar/stellar-core/pull/4131
• [x] stellar-xdr cli – will release with latest stable, and is already pinned as so.
• [x] soroban-rpc (in soroban-tools) – will stick with latest stable https://github.com/stellar/soroban-tools/pull/1161
• [x] soroban-cli (in soroban-tools) – will stick with latest stable https://github.com/stellar/soroban-tools/pull/1161
• [x] soroban-examples
  • [x] is already pinned to stable and should stay on stable to minimize need for new devs to install old rusts
  • [x] should have an msrv test added so that a wider variety of rusts are confirmed supported (equal to the sdk's range) for easier onboarding https://github.com/stellar/soroban-examples/pull/293
• [x] soroban-example-dapp
  • [x] should pin to stable to minimize need for new devs to install old rusts https://github.com/stellar/soroban-example-dapp/pull/141
  • [x] should have an msrv test added so that a wider variety of rusts are confirmed supported (equal to the sdk's range) for easier onboarding https://github.com/stellar/soroban-example-dapp/pull/140

Libraries will need updating using the same pattern that is in use in https://github.com/stellar/rs-stellar-xdr/pull/341:

• [x] rs-stellar-xdr https://github.com/stellar/rs-stellar-xdr/pull/341
• [x] rs-soroban-env https://github.com/stellar/rs-soroban-env/pull/1319
• [x] rs-soroban-sdk https://github.com/stellar/rs-soroban-sdk/pull/1205
• [x] soroban-tools https://github.com/stellar/soroban-tools/pull/1159
• [x] rs-stellar-strkey https://github.com/stellar/rs-stellar-strkey/pull/61
• [x] bytes-lit https://github.com/stellar/bytes-lit/pull/37
• [x] crate-git-revision https://github.com/stellar/crate-git-revision/pull/20
• [x] escape-bytes https://github.com/stellar/escape-bytes/pull/7

[leighmcculloch]

I suggest for anything that isn't stellar-core, we should probably leave the rust-toolchain.toml files set to stable and allow them to be released with latest stable. They don't carry the same divergence risk (except maybe preflight to some degree) but all our Rust project are retested before release in any case.