CI: replace "uses" in GHA, unless needed for CI interaction

[mvglasow]

As mentioned in another issue, the GHA workflow makes use of uses in various places to reference another GHA workflow. This is limiting in terms of portability and extensibility – including migration to a different site (#445, #705) or cross-compilation (#443, or the idea of an Android port we discusses on the mailing list).

TL;DR: uses: ocaml/setup-ocaml@v2 and uses: ilammy/msvc-dev-cmd@v1 should be replaced with something more portable. Other occurrences of uses: are for interaction with the CI environment.

A quick cat .github/workflows/CI.yml | grep "uses:" | sort -u gave me:

uses: actions/checkout@v2
uses: actions/checkout@v3
uses: actions/download-artifact@v2
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v2
uses: ilammy/msvc-dev-cmd@v1
uses: ocaml/setup-ocaml@v2
uses: softprops/action-gh-release@v1

The ones in actions and softprops, as I understand it, interact with the CI environment (git clone the repo, store job artifacts, retrieve previously stored artifacts, publish artifacts for released versions). That means we need them as long as we are on GHA, but any other environment would offer something equivalent.

My concern are ilammy/msvc-dev-cmd and especially ocaml/setup-ocaml. These are steps that could (and should) be CI-independent. I see two ways to resolve those:

1. Replace each action with the equivalent sequence of OS shell commands.
2. Use a Docker image (container.image) which has the necessary components on board (or create one).

Option 1 is the most portable (even to a VM in a CI container, e.g. to build for non-x86 platforms) but comes at a performance penalty, as we’d be installing dependencies from scratch once per CI run. Option 2 is faster as we’d be reusing the same Docker image with everything we need already on board, but less portable to anything other than Docker (such as a VM).

A closer look at the CI script and build matrix tells me that ilammy/msvc-dev-cmd is only needed when the build matrix specifies an OCaml version ending in +msvc, but currently there is no such entry in the matrix. In other words, this looks like it’s dead code at the moment – unless there are plans to do MSVC builds in the future.

As for ocaml/setup-ocaml, unrolling this into shell commands might be more complex than it needs to be – it runs on node16 (not a standard OS shell) and runs two .js scripts, each several megabytes in size.

A quick and dirty workaround would be to instead run on a Linux distribution (and version) which bundles our target OCaml version, then install everything we need via apt. Downside is we might be limited in our choice of OCaml versions. A cleaner, but more complex, solution is to install the desired OCaml version by hand (with which I, admittedly, don’t have any experience.)

[mvglasow]

I’ve started experimenting around at https://github.com/mvglasow/unison/tree/portable-ci – feel free to take a look.

At least the Linux build works with OCaml installed as documented at https://ocaml.org/docs/up-and-running#installation-for-unix (basically, install opam from the OS repositories, then use opam to install the OCaml build environment). I am only gradually including other jobs.

Now I just need to figure out how to do this for the other platforms: do I need to use brew or port on MacOS? On Windows, can I install opam via the Cygwin installer – or is there any requirement that rules this out?

[mvglasow]

Update: port does not seem to be available on MacOS, whereas brew works – although opam init fails with an error (see bug report at https://github.com/ocaml/opam-repository/issues/23358).

After installing opam on cygwin, the binary is not found. Still investigating...

[mvglasow]

Strange, on Cygwin opam 2.0.7-1 is getting installed, and according to the package, the binary should be placed in /usr/bin. However, ls doesn’t find it there, and I cannot run it – neither without a path (/usr/bin is definitely on the path) nor with an explicit path. No idea what’s going on here.

[mvglasow]

Apparently GHA defaults not to Cygwin’s version of bash, but to git-bash (which comes bundled with the native version of git for Windows) – which obviously doesn’t know anything about Cygwin. So I need a way to get the real thing.

Also, the cygwin setup is interesting. The image already seems to have a setup in C:\cygwin64, but the external GHA has a hardcoded location of D:\cygwin, and our script installs some additional components there. We might no longer need this after migrating to a native install script.

[tleedjarv]

A few quick notes.

• vast majority of the GHA CI script by line count is for
  • GHA infra setup
  • Windows (setting up env, building and packaging) -- this is double-bad as the GHA infra greatly increases the complexity needed for setting up the Windows env
  • inline patches for building old versions with recent compiler
• ilammy/msvc-dev-cmd@v1 is currently not used because building with MSCV is very complex or almost impossible in GHA. But it does seem to work: https://github.com/bcpierce00/unison/wiki/Building-Windows-binary-with-MSVC-Ocaml-4.12-Using-Docker-Container so getting it working in CI would be a bonus (but good luck finding a CI provider)

Building Unison from scratch is actually very simple. This is all that is needed: First build OCaml compiler (unless you get the required version from your pkg repository)

./configure
make
make install

Building unison

make

Yes, it's that simple. (It even works in Windows if you have Cygwin and MinGW installed.)

The complexity comes from: lablgtk (for GTK GUI), and Windows. Note that Mac native GUI does not require lablgtk.

Building lablgtk from scratch is probably easiest with OPAM but once opam is involved, everything becomes more complicated. In Windows you're lucky to get anything working at all. In macOS you probably need brew. In theory, this could work:

# install opam somehow
opam create switch <compiler-variant-version>
opam install lablgtk3
# in Unison source dir
opam exec -- make

alternatively

# install or build OCaml compiler
opam create switch <system-ocaml-compiler>
... rest same as before

That's the theory. You can see from the GHA CI script that it's not that simple in practice.

And one last comment: building both the OCaml compiler and Unison sources (+ manual) can be much simpler without GHA; much of the complexity is due to GHA itself (as you've already noticed).

[mvglasow]

Windows (setting up env, building and packaging) -- this is double-bad as the GHA infra greatly increases the complexity needed for setting up the Windows env

For the moment, I’ll change the OCaml setup part for Ubuntu only. On the long run (once all other issues are sorted out) it should be the same on all platforms, with only the first line (running the OS package manager) being different.

One thing we might be able to look into is using a different Windows image from Dockerhub, which would also be a step towards portability. Just any image that doesn’t come with a non-Cygwin bash. Though my practical experience with that is limited to Gitlab+Linux – the GHA+Windows combo might well come with obstacles I didn’t have on my radar.

ilammy/msvc-dev-cmd@v1 is currently not used because building with MSCV is very complex or almost impossible in GHA

Just curious, how important is being able to use MSVC instead of gcc? In any case, if that’s currently not being used, I’d leave that out for the moment. The wiki article mentions a Docker container, has anyone tried using that in GHA?

[mvglasow]

More question marks on Ubuntu:

• opam switch create ${{ matrix.job.ocaml-version }} works on 20.04+, but not on 18.04 – I get opam: Invalid switch subcommand "create". Do I need a different syntax on older versions?
• For build, the GHA sets opam-pin and opam-depext to false, However, lablgtk install runs opam depext. This fails if I don’t install depext but works with the GHA, although I don’t see it installing depext anywhere. What’s up here?

[mvglasow]

And another oddity on build for Windows: all of a sudden ocaml/setup-ocaml@v2 fails because curl is not found – although for Windows, nothing should have changed on this job. The commit in question is https://github.com/mvglasow/unison/commit/90677599ae818d12e0fbc0e4bed4a8e7cc6134e5, and the CI workflow is at https://github.com/mvglasow/unison/actions/runs/4217403442.

Just noticed that master seems to be having this issue as well – this strongly indicates an issue unrelated to my changes.

[tleedjarv]

Just curious, how important is being able to use MSVC instead of gcc? In any case, if that’s currently not being used, I’d leave that out for the moment.

I don't know, actually. MinGW seems to work perfectly well. Apparently the real issue with MSVC vs MinGW is ABI compatibility or something like that. I don't build on Windows myself and I don't know anything about MSVC or MinGW, so I'd appreciate someone more knowledgeable explain this or just make a decision "we go with X for reasons Y".

The wiki article mentions a Docker container, has anyone tried using that in GHA?

Not that I know of. Is that even possible?

opam switch create ${{ matrix.job.ocaml-version }} works on 20.04+, but not on 18.04 – I get opam: Invalid switch subcommand "create". Do I need a different syntax on older versions?

No idea (don't use opam myself) but it sure seems like it. I think it would be easier to just install a newer opam since 18.04 most likely comes with a really outdated version.

For build, the GHA sets opam-pin and opam-depext to false, However, lablgtk install runs opam depext. This fails if I don’t install depext but works with the GHA, although I don’t see it installing depext anywhere. What’s up here?

I think you can just skip anything depext because opam versions >= 2.1 should automate this. If it doesn't then you probably have an outdated version of opam.

And another oddity on build for Windows: all of a sudden ocaml/setup-ocaml@v2 fails because curl is not found – although for Windows, nothing should have changed on this job. Just noticed that master seems to be having this issue as well – this strongly indicates an issue unrelated to my changes.

Yes, this is a bug with GHA cache. You can temporarily fix it by deleting any GHA cache with "cygwin" in the name.

[mvglasow]

Docker container

Is that even possible?

Apparently, though that feature seems well-hidden and not very widely used on GHA (unlike CircleCI or Gitlab) – didn’t know myself until I mentioned somewhere that apparently GHA does not support this, and got corrected. See container.image.

I think it would be easier to just install a newer opam since 18.04 most likely comes with a really outdated version.

Which brings me to the next question – is there a need to build on a release of Ubuntu that’s almost five years old and two LTS versions behind? I see the point for building against different OCaml versions (due to dependencies when Unison 2.48–2.51is involved on the other end). As for the Linux distribution and version, the only requirement I can think of are dependencies, which should only be an issue with non-static builds.

I think you can just skip anything depext because opam versions >= 2.1 should automate this. If it doesn't then you probably have an outdated version of opam.

lablgtk install fails when I skip depext on 20.04 and 22.04 (the latter comes with opam 2.1.2-1). On 18.04 I don’t even get that far.

[gdt]

In my view the main point of CI is CI, not building binaries, and thus building on older systems is reasonable if we expect unison to build in that environment.

[tleedjarv]

And another oddity on build for Windows: all of a sudden ocaml/setup-ocaml@v2 fails because curl is not found – although for Windows, nothing should have changed on this job. Just noticed that master seems to be having this issue as well – this strongly indicates an issue unrelated to my changes.

Yes, this is a bug with GHA cache. You can temporarily fix it by deleting any GHA cache with "cygwin" in the name.

It seems that setup-ocaml has released a workaround for this issue. Windows builds should be working again.

[tleedjarv]

Docker container

Is that even possible?

Apparently, though that feature seems well-hidden and not very widely used on GHA (unlike CircleCI or Gitlab) – didn’t know myself until I mentioned somewhere that apparently GHA does not support this, and got corrected. See container.image.

Also Windows?

I think you can just skip anything depext because opam versions >= 2.1 should automate this. If it doesn't then you probably have an outdated version of opam.

lablgtk install fails when I skip depext on 20.04 and 22.04 (the latter comes with opam 2.1.2-1). On 18.04 I don’t even get that far.

I have no more knowledge about this, only that I can see in the CI logs:

opam-depext is unnecessary when used with opam >= 2.1. Please use opam install directly instead

Of course, yet another path is to just not use depext, instead directly invoke system package manager to install GTK. This is maybe a bit tighter coupling to the specific build environment than we'd like but doesn't seem too bad (assuming that whatever package manager is used takes care of transitive dependencies).