Selecting compatible versions for `forc` and `fuel-core` for the default toolchains (e.g. stable, nightly, latest)

[mitchmindtree]

In the common case that the user hasn't specified a custom toolchain with specific versions, we need to decide on which version of forc (and associated plugins) and fuel-core to select.

I suggest that we select the version of fuel-core based on the selected version of forc, as forc depends on fuel-core, but the inverse is not true.

One way of achieving this could be to select the version of fuel-core based on the version that was used within forc's CI within the commit associated with the selected forc version. We could do this by doing something like:

1. Add a tiny file to the root of the Sway repo called fuel-core-version, which just contains the semver version and nothing more.
2. Use this file to specify the version of fuel-core to fetch in the Sway CI.
3. After fetching forc, determine the version of fuel-core to use by reading the fuel-core-version file.

An alternative could be to select the version based on the timestamp of the commit (e.g. select the fuel-core commit with the first timestamp that precedes the timestamp of the forc release), but I think the concept of selecting commits by similar timestamps is probably much less robust than the known, CI-tested version.

[adlerjohn]

One problem with relying on either sway or fuel-core to specify this is that it doesn't account for new releases, only current releases. E.g. a version of fuel-core (the binary) might be bumped that only breaks fuel-core (the library), but the VM is the same. In such a case, a particular release of forc won't be able to capture that it does in fact work with this new release of fuel-core.

The file metadata could instead be stored on a directory in this repo.

[mitchmindtree]

One problem with relying on either sway or fuel-core to specify this is that it doesn't account for new releases, only current releases. E.g. a version of fuel-core (the binary) might be bumped that only breaks fuel-core (the library), but the VM is the same. In such a case, a particular release of forc won't be able to capture that it does in fact work with this new release of fuel-core.

In order to ensure we deliver the latest patch release, we could treat the specified fuel-core-version from the Sway repo in the same way that cargo treats semver versions of registry dependencies when generating a new lock file. That is, given a full version <major>.<minor>.<patch>, pick the latest patch release for the given major minor version that is at least <patch> or greater.

E.g. if fuel-core-version specifies 1.2.3 but 1.2.7 has been released with new patches, 1.2.7 is selected instead.

The file metadata could instead be stored on a directory in this repo.

By this do you mean storing the version sets in this repo?

My concern with this approach is that we lose the main benefit of using the sway repo for this, which is that it already has a massive set of E2E tests. If we're to define version sets in a file in this repo, then we'll also want to introduce some extensive integration testing that gets run for each of the version sets as a part of fuelup's CI. I see guaranteeing compatibility between the selected set of versions as one of the few main features of fuelup.

It's also not clear to me why it would be any easier to keep versions up to date in this repo than it would be to keep the fuel-core-version up to date in the sway repo? One way we could ensure a fuel-core-version file in the Sway repo stays up to date could be to have some CI job that periodically (or perhaps in response to some fuel-core release hook) checks fuel-core-version against the latest published version of fuel-core and either warns or automatically opens a PR to bump the version if it's out of date.

I do empathize with the desire to keep the version selection responsibilities contained to this repo, particularly as ideally the Sway repo should not need to accommodate or even know about fuelup's existence. However, the E2E testing in that repo is difficult to pass up and if we can get the benefits of that testing with the addition of one semver file in the Sway repo, perhaps it's worth it?

[adlerjohn]

then we'll also want to introduce some extensive integration testing that gets run for each of the version sets as a part of fuelup's CI

yes.jpg. We need those anyways!

The problem about semver in the Sway repo is that fuel-core is used as both a library and a binary. We can end up missing minor releases that are binary- but not library-compatible.

[mitchmindtree]

We can end up missing minor releases that are binary- but not library-compatible.

True, I see where you're coming from. If that's the case, and we continue to treat our crate semver versions as if they are only meaningful w.r.t. breakage at the binary-user level (as we currently do for forc et al in the Sway repo), neither ourselves or other users of our libraries will have any meaningful way to reason about their versioning in general. I've opened https://github.com/FuelLabs/sway/issues/1950.

That said, I'm realising this still doesn't solve the issue we have here, which is that it's difficult to practically infer the fuel-core binary version from the forc binary version - the above just makes the impracticality of this clearer!

---

I agree, it seems having an index of the version set for each toolchain in this repo might be the best way to go.

How does the following sound:

Toolchain	Versions
nightly	Builds triggered by a nightly chron-job (can be done with GitHub actions like this) from master once-per day. Versions are updated if integration tests pass for all binaries, otherwise the last successful build continues to be used. Mostly for use by fuel devs or app devs that need cutting edge features.
latest	Uses the latest published versions of each binary crate. Attempt to update the index upon any of the binary crates we care about being published. Indexed versions are only updated if all integration tests pass for all binaries, otherwise the last successful set continues to be used (should never happen for patch releases).
stable	??? Not sure how to define this yet in a way that is actually more "stable" than latest?

To keep things simple to begin, we can just start with latest.

The index itself could just be some toml file with a table of versions for each tool.

We'll also want to automate the updating of this index somehow, either using actions or some other GitHub bot. I'm not sure on the best approach here, ideally we'd do this with actions so that all the activity is publicly observable.

I'm not sure what the most efficient way of triggering fuelup index updates when forc or fuel-core publishes a new release. Maybe github actions will let us to hook into actions from other github repos somehow.

@binggh thoughts? Does this sound like enough to get started?

[eightfilms]

Yup, all in all sounds good for a start for at least the latest version. What I'm working on in #73 will support the toolchain pattern like in rustup instead of simply installing all the bins as we do now into .fuelup/bin, which was good for a start but not enough to support where we want to go with the toolchain pattern.

I agree, it seems having an index of the version set for each toolchain in this repo might be the best way to go.

Rust seems to host channels on their dist server as a .toml file (eg. https://static.rust-lang.org/dist/channel-rust-nightly.toml) and I think we can probably do the same by publishing to fuelup GitHub pages as well.

Also, if I'm understanding this right, since we're publishing all the version sets in this repo, fuelup will query this index and fetch the correct version for their supported architecture/os instead of having to build the URL manually within fuelup, which is neat!

Indexed versions are only updated if all integration tests pass for all binaries

Would there ever be scenarios where we release binaries while integration tests fail?

??? Not sure how to define this yet in a way that is actually more "stable" than latest?

This was also where I was having difficulty trying to define, since technically we are still very early and introducing breaking updates pretty often. Maybe this is a problem for future us but setting up the infra to support latest first then nightly will allow us to handle stable pretty easily when we decide what stable means to us.

I'm not sure what the most efficient way of triggering fuelup index updates when forc or fuel-core publishes a new release. Maybe github actions will let us to hook into actions from other github repos somehow.

This would have to be a push-based action from fuel-core and the sway repo into this repo right? I briefly tried to find something within GitHub actions page and seems like workflow dispatch or repository dispatch might fit the bill. Seems like we just need a workflow on the other repositories sending some sort of payload (eg. version numbers of forc/fuel-core) to this repository upon release, and on here we can have a workflow to receive the payload, update and publish the new index.

[mitchmindtree]

Would there ever be scenarios where we release binaries while integration tests fail?

I don't think so - if we can't guarantee that the set of versions work together, I think we're breaking the major promise of fuelup.

I think this is also how nightlies work for Rust too. E.g. I remember in the early days there could be up to a week or two gap in nightly releases in the case that there was some issue on master that couldn't be resolved for whetever reason. Obviously we want to avoid this, but I think the most important thing is that the set of versions under each toolchain work together, at least under whatever integration testing we can come up with.

Maybe this is a problem for future us but setting up the infra to support latest first then nightly will allow us to handle stable pretty easily when we decide what stable means to us.

Agreed - we might even just decide that latest should be stable! But yes, let's leave that for future us.

This would have to be a push-based action from fuel-core and the sway repo into this repo right? I briefly tried to find something within GitHub actions page and seems like workflow dispatch or repository dispatch might fit the bill. Seems like we just need a workflow on the other repositories sending some sort of payload (eg. version numbers of forc/fuel-core) to this repository upon release, and on here we can have a workflow to receive the payload, update and publish the new index.

:+1: sounds decent to me! Ideally the CI in the other repos wouldn't have to know about fuelup and we could instead "watch" them or subscribe to their actions or something from this repo's CI to keep the separation of concerns distinct. But if it turns out the most convenient/conventional way is to push from the other repos, that sounds fine too as ultimately we're in control of all of them.

[eightfilms]

there could be up to a week or two gap in nightly releases

Ah okay, you're talking about nightly - I was confused for a moment since in the case of latest, we would be publishing whenever a new release is out once CI tests passed.

Ideally the CI in the other repos wouldn't have to know about fuelup and we could instead "watch" them or subscribe to their actions or something from this repo's CI to keep the separation of concerns distinct

Hmm, that was my first intuition as well for fuelup to pull from the other repos so that the other repos are (rightfully) ignorant of fuelup - didn't seem to find something for now but I'll look further into it soon!

[eightfilms]

Going to prioritise and work on this - testing workflows within the dummy CI test repo.

[eightfilms]

Ideally the CI in the other repos wouldn't have to know about fuelup and we could instead "watch" them or subscribe to their actions or something from this repo's CI to keep the separation of concerns distinct.

What do we think about a cron workflow here checking the latest tags through the GitHub API on the other repos? The downside here would be that depending on the interval of the checks we might not have the latest versions right away through fuelup, which seems like a non-issue to me, unless it's critical that the latest releases are ready through fuelup right away once they are released. Also I'm not aware of the CI runtime limits - would it be sane to have this cron workflow run every, say, 15 minutes?

[mitchmindtree]

What do we think about a cron workflow here checking the latest tags through the GitHub API on the other repos?

Hmm this does sound a bit hacky - would be greatly preferred if we can do this with a proper hook or event or something along those lines to remain immediate. I can also foresee there being occasions where app devs go to use a new release of forc immediately after it's published at the Sway repo and wonder why they don't have the latest version still :smiling_face_with_tear:

[eightfilms]

I didn't seem to find a way to 'subscribe' to another repo's releases, hence the hacky suggestion :/

The alternative is to settle for a push-based action to update versions here then, which isn't exactly pretty as well since the other repos would now have to be aware of fuelup. You could argue that if the binary packages' main form of distribution is through fuelup then it would be fair for them to be aware of fuelup anyway, and that the impetus is on those repos to push the correct versions into a .toml file in this repo...

[eightfilms]

I created 3 smaller issues and hope to use this as the tracking issue, since this issue seems to encompass the entire version set selection problem that we have to tackle across the different toolchains, each of which will probably be dealt with differently (eg. latest is published whenever a latest release is published, nightly is only published when master builds and passes integration tests, and while the meaning of stable has yet to be decided we likely want a different way to tackle that issue as well)

[eightfilms]

Closing this in favour of #440 to narrow the scope and for more contextual info, since 2/3 of the toolchains mentioned here are live.