Support project specific toolchains with `fuel-toolchain.toml`

[eightfilms]

Following some conversations on changes in forc from 0.14.5 -> 0.15.1 causing SwaySwap to break, I think there's space for fuelup to support usage of project-specific toolchains (which is basically rustup's override mechanic, described here).

Will expand on this when I have more thoughts regarding this but my preliminary idea is that, instead of having a rust-toolchain.toml, we can have an entry within Forc.toml (would this create issues with parsing the toml?) stating which version of forc is built with the project. fuelup can detect this and decide which version of forc to run for that project.

I think this idea is also mentioned here in the original issue created by @adlerjohn.

We would need to support the toolchain pattern #63 for this.

cc: @mitchmindtree @Braqzen for comments and ideas

[Braqzen]

Given that we follow how cargo does things it's probably better to continue to use the same structure namely have a separate file for configuration. There may be additional components that are added to this configuration file over time and it would be less work to append to the config rather than parsing out of Forc.toml. Moreover, I think it's conceptually easier to distinguish between a generic config file and something specific to building a project

[adlerjohn]

We also have to consider how this interplays with workspaces, since we want to intentionally depart from Cargo's behavior for workspaces.

[mitchmindtree]

+1 to @Braqzen's point of keeping the separation of concerns between Forc.toml and fuelup-toolchain.toml distinct. While we want fuelup to be as easy as possible to use, there will still be many folk who will prefer to use their distribution's package manager or some other approach to packaging their project.

[eightfilms]

Following a discussion with @mitchmindtree, it seems like we can emulate a part of rustup with their rust-toolchain.toml/rust-toolchain file, where the file provides info regarding the specific version of a toolchain to run a Rust project. The TOML file looks like this:

[toolchain]
channel = "nightly-2020-07-10"
components = [ "rustfmt", "rustc-dev" ]
targets = [ "wasm32-unknown-unknown", "thumbv2-none-eabi" ]
profile = "minimal"

In Rust, channel can also be replaced by path, but we can probably omit path since it seems like it's for individual use - I couldn't imagine a workflow where devs would want to install toolchains and specify paths separately. A TOML file like this placed within the root of a project directory would be a good way to freeze/lock a toolchain for a project.

[eightfilms]

Updated proposed look of fuel-toolchain.toml:

[toolchain]
channel = "latest"

This pins the current project toolchain to latest.

the TOML also accepts an optional table components, if you wish to override certain components from the toolchain:

[toolchain]
channel = "latest"

[components]
forc = "0.31.3"

This means that all other components will be derived from the latest toolchain.

[mitchmindtree]

As discussed, we also need to ensure toolchains are pinned to a specific channel version. E.g. nightly will be easy as we can use the date:

[toolchain]
channel = "nightly-2022-12-20"

Perhaps we can use a date for latest too, i.e.

[toolchain]
channel = "latest-2022-12-20"

but in the case there were multiple latest updates in one day (very possible) we can allow for specifying a trailing index? E.g. the following would refer to the 3rd latest release on that date?

[toolchain]
channel = "latest-2022-12-20@3"
# or
channel = "latest-2022-12-20-3"

In the case that there were multiple releases in one day, but the specific release wasn't specified, perhaps we can assume the first of that day?

@bingcicle I know we discussed using an every increasing index for the latest channel that increments on every release - I'm still open to this. I thought I'd mention the date approach is that it might be more useful for readers of the fuel-toolchain.toml to easily see when the toolchain was pinned?

[eightfilms]

I left specifying the dates out of this issue for now because I was going to create a new issue and PR for it following #317 since it might require more thought - will cross-post your comment there

[eightfilms]

Though I will say that even with the date tagged to the channel, we will probably still need some form of lock or index file. I don't think forcing devs to specify the date explicitly would be good DevEx.

First off, when thinking about how devs will interact with fuel-toolchain.toml, there are probably 2 ways:

1) publishing or updating it to tell other devs working on the same project that the toolchain changed, or 2) reading it to download the components needed to run a project.

IMO the feature we ship here should aim to make these 2 steps easy.

Specifying a date will easily fix the issue with number 2 - we tweak the CI a bit to save previous published latest TOML files in an archive and let fuelup download from the archive, and we're done.

For number 1 - on a day-to-day basis while working on projects with a toolchain you would not know or care what date the latest toolchain is published at. This means that devs would have to take a detour by either checking what is published on our gh-pages branch directly, or using a command that we must ship to show publish dates, before they can specify the channel, and adds a lot of ambiguity to the setup process (Am I currently using the most up-to-date toolchain? What does latest-2022-12-21 mean?) This is probably fails number 1 :slightly_frowning_face: You could say that we could fix this by making it so that the toml is generated from a command, but it doesn't stop people from editing it themselves.

I think fuelup should probably handle all these implicitly behind the scenes.

[mitchmindtree]

I left specifying the dates out of this issue for now because I was going to create a new issue and PR for it following https://github.com/FuelLabs/fuelup/pull/317 since it might require more thought - will cross-post your comment there

I don't think we should consider landing this feature without some form of locking. In my mind, the locking-to-a-specific-set-of-tools is the main purpose of this feature - to get closer to ensuring that others building the project can do so with a known set of working tools.

I don't think forcing devs to specify the date explicitly would be good DevEx.

on a day-to-day basis while working on projects with a toolchain you would not know or care what date the latest toolchain is published at

For these reasons, I was imagining that the more common approach would be to generate this file using a rustup command that uses the currently installed toolchain. That said, I also don't think specifying a date is so bad? E.g. we could have fuelup show output what it currently is. That said, I'm open to going the lock file route instead, but I guess the cost would be extra file noise in the repo.

1. publishing or updating it to tell other devs working on the same project that the toolchain changed, or
2. reading it to download the components needed to run a project.

IMO the feature we ship here should aim to make these 2 steps easy

Hmmm in my mind, the main purpose for this file is for fuelup to be able to automatically fetch and use known, working versions for each of the tools for the project. I'm logging off for the day now, but maybe we can have another chat on this tomorrow?

[eightfilms]

Finalized version of how this might look:

We will only allow specifying toolchains with dates, eg. latest-2022-12-22.

Reason: just latest is not good enough, as we cannot guarantee that everyone using the toolchain toml will have the same latest toolchain on their local setup unless they religiously run fuelup update to keep everything up to date. There is also not much point in keeping a project pinned to the latest toolchain alone, because that still doesn't guarantee that every dev sharing a project will share the same components.

Sample TOML:

[toolchain]
channel = "latest-2022-12-22"

PR #317 will be updated to support this format instead.