Support next tag

[josephjclark]

The PR adds support for the @next tag in the Worker and CLI.

Background

I am working on pr-release dist-tag for adaptors, making them available to the worker in particular but only if asked for.

Over in adaptors, I've added logic to build and publish prereleases with the @next tag.

Implementation

Here in kit, I've had to add logic to the CLI, Runtime and Worker to better manage @next - and for that matter, @latest.

• Always lookup versions when @next or @latest is passed (but use cached version if still up to date)
• Return mapped specifier list from the runtime's install function - because @next needs converting to a version number and passing to the compiler and runtime

The change is small but deep, so I'm testing carefully before I'm ready to release.

Test Notes

The integration tests are destined to be a little bit fragile as they'll depend on specific adaptor versions being available on NPM.

I am trying to decide whether to skip these tests or do consolidate releases before merging.

QA

In the CLI:

• Install openfnx
• Either clean your repo openfnx repo clean or pass a --repo-dir=/tmp/my-new-repo (whatever path you like) each of these commands
• Run a basic job with -a common@1.7.7
• Now run with -a common (no version number) and --log debug. Logs should tell you it's using the latest installed version of common (1.7.7)
• Now run with -a common@latest. The CLI should install the latest version of common (>1.14.0)
• Run with -a common@latest and the CLI should tell you its checking versions, choosing the latest, and not re-installing it
• Now run the same job with a test adaptor -a test-tmp (fn is available, also version which is fun for this). The CLI should install the latest version 1.0.1-next.0
• Run with -a test-tmp@next, which should install the latest, pre-release build of the adaptor
• Run with -a test-tmp@next and it should check the latest version, but do nothing

Phew.

In Lightning there isn't much to test yet.

When you pick @latest from the drop-down menu, Lightning will send the actual version number, not @latest, to the worker. In other words lightning is in total control of the versions.

I will later raise a PR against Lightning to enable @next, if it exists.

[josephjclark]

Still something wrong with the engine and @latest. Need to test that carefully tomorrow

[josephjclark]

Oh my days it's passing!! I wonder if this is actually a reasonable fix - struggling to get my head back into this space. I'll self-review this afternoon

[josephjclark]

The behaviour in CLI in production (validated through testing) is:

• If you don't pass a version, you'll get the latest installed version from the repo. If nothing, we autoinstall @latest
• If you pass @latest it'll use latest as a version string and fetch the latest

This is a problem because now and forever, @latest will be locked in that repo to whatever version it downloaded. If we release a new common now, my CLI won't use it even if I pass @latest.

So that's actually not very good!

The behavior on this branch is:

• If no version is passed, we use the lasest available
• @latest we check the actual latest and install if needed
• @next we check the actual next and install if needed

Is this the same in CLI AND Worker?

I think that when Lightning runs from @latest, it instantly version locks the number to whatever is latest now. That's the correct behaviour but I need to verify this.

With @next, though, Lightning should literally use next as the version label, and trust the worker to track the rolling head.