Setting continuous integration

[amirouche]

Can we setup continuous integration at, say Cirrus CI, since it seems like the most advanced CI system.

ref: https://mailman.iro.umontreal.ca/pipermail/gambit-list/2019-November/009222.html

[lassik]

On the bright side, FreeBSD passed easily!

[lassik]

Note that since FreeBSD and MacOS do not work inside Docker containers, they use some different kind of virtualization. But those test cases do not fail on FreeBSD. Maybe it's intermittent failure? We've only had one macOS test run so far.

[lassik]

There's now a Cygwin build: https://github.com/udem-dlteam/gambit/blob/lassik/cirrus/.cirrus.yml

Do you know why it's trying to run autoconf when the make command is issued? https://cirrus-ci.com/task/5241569640185856

[feeley]

Nice! The issue may be related to incorrect modification dates on the files? Could you ls -l config* just after cloning?

[lassik]

-rwx------+ 1 ContainerAdministrator ContainerAdministrator  44166 Nov  6 14:09 config.guess
-rwx------+ 1 ContainerAdministrator ContainerAdministrator  36141 Nov  6 14:09 config.sub
-rwx------+ 1 ContainerAdministrator ContainerAdministrator 455587 Nov  6 14:09 configure
-rwx------+ 1 ContainerAdministrator ContainerAdministrator 114207 Nov  6 14:09 configure.ac

[feeley]

Maybe the issue shows up because of the coarse time stamp granularity on Windows, which is 2 seconds I think. So perhaps make can't tell that configure is newer than configure.ac. Could you add to the CI script a touch configure to force a newer time stamp?

[lassik]

It's only 2 seconds on the old FAT file systems; NTFS timestamps use "100-nanosecond intervals".

[feeley]

OK... but can you still try adding a touch? If that doesn't work then it might be a deeper problems with the makefiles or cygwin's make.

[lassik]

MacOS passed in the latest build. That means the failures in the thread tests are intermittent.

[lassik]

Since this issue is related to core Gambit instead of libs, can you move it into gambit/gambit?

[lassik]

touch configure fixed the issue of Cygwin trying to run autoconf.

[lassik]

Also on Cygwin:

checking ws2tcpip.h usability... no
checking ws2tcpip.h presence... yes
configure: WARNING: ws2tcpip.h: present but cannot be compiled
configure: WARNING: ws2tcpip.h:     check for missing prerequisite headers?
configure: WARNING: ws2tcpip.h: see the Autoconf documentation
configure: WARNING: ws2tcpip.h:     section "Present But Cannot Be Compiled"
configure: WARNING: ws2tcpip.h: proceeding with the compiler's result
configure: WARNING:     ## -------------------------------------- ##
configure: WARNING:     ## Report this to gambit@iro.umontreal.ca ##
configure: WARNING:     ## -------------------------------------- ##
checking for ws2tcpip.h... no

[lassik]

All platforms green! https://cirrus-ci.com/build/6149589735833600

[feeley]

Wonderful! So Cirrus appears to suit our needs.

Can you also add builds with mingw and Microsoft compilers? And also build status feedback for the README badges? With Travis and Appveyor I also was building with C++ (to make sure Gambit builds with either a C or C++ compiler), and also with --enable-multiple-threaded-vms (to test the parallel GC), and also with --enable-ansi-c (to test a bare-bones build).

When all of this is supported I'll accept a pull request to switch Gambit to Cirrus CI.

Separately, and likely after switching to Cirrus CI, we can add some settings such as the universal backend and the native backend. That will be many settings to test! But it will be a good way to avoid bitrot with those backends.

[lassik]

Can you also add builds with mingw and Microsoft compilers?

We can do that. Gambit's current AppVeyor config has:

    - arch: "x86-64"
      tools: "mingw"

    - arch: "x86-64"
      tools: "msvc"

    - arch: "x86-64"
      tools: "vstudio.bat"

    - arch: "x86-64"
      tools: "cygwin"

    - arch: "x86-32"
      tools: "msvc"

    - arch: "x86-32"
      tools: "vstudio.bat"

Of these, Cirrus currently covers just 64-bit Cygwin. I wonder what are the easiest ways to install the requisite toolchains for the other builds. The Windows image provided by Cirrus comes pre-installed with the Chocolatey package manager (choco command) so that may be the easiest option. Scoop will probably be easy to install if needed. I'm not sure how to install MSVC; it may be pre-installed.

Chololatey has at least one MingW package: https://chocolatey.org/packages/mingw

[lassik]

The very portable libarchive is using Cirrus, so we can adapt what they do: https://github.com/libarchive/libarchive/tree/master/build/ci/cirrus_ci

[feeley]

Yes that build script for the choice of Windows dev tools seems well written. My only comment would be to use the ".bat" extension instead of ".cmd" (".bat" is used in many places in Gambit and it is easier to grep if only one extension is used).

[lassik]

The MSVC compiler is cl but what do we use for make - should we install GNU make from MingW, or does Visual Studio ship with a version of make that works with Gambit's autotools?

[lassik]

Would it make sense to run make install as part of CI builds, and roll the installation directory into a tar file that is shipped as a build artifact? That would be a really easy way for people to install and test cutting-edge builds of Gambit on popular platforms without compiling anything. It would also be an easy way to get a gsc-boot binary for compiling the master branch.

[lassik]

We could do something like ./configure --prefix=/usr/local/Gambit/$branch (as discussed earlier in this issue) and then tar -czf gambit-$branch-$commithash.tgz and tell Cirrus to preserve that tar file.

[feeley]

The only problem is that the directory in which the tar file is untared must be known when the CI build is done. The DESTDIR environment variable could be used if the install path on the CI machine and the target machine don't match.

[lassik]

Yeah, users would have to use the same directory locally. $HOME/.local/Gambit/... would be a good location but since $HOME varies per user, I assume it doesn't work.

The trouble with the CI artifacts is we don't know anything about the target machine since any number of users might download the tar files onto different machines/user accounts. We'd to agree on a convention.

[feeley]

That would be useful, but it should be seen as a "fallback" and not be part of the standard way of building Gambit (otherwise building Gambit will depend on CI, which is kind of backwards).

[feeley]

I don't recall, but maybe the installation directory can be relative to the user's home (if it contains a ~/ at the start).

[feeley]

That doesn't work because the configure script insists on an absolute path.

[lassik]

Agreed. I meant the CI tarballs as an easy alternative to building from source, not as a replacement. Some projects (operating systems, etc.) that are complex to buil publish frequent binary snapshots. People who want to follow cutting-edge stuff but don't need the up-to-the-minute latest source code can use those, sparing the effort of compiling it themselves. I ran OpenBSD snapshots back in the day, as did many other people, and it was a good experience. Nowadays CI is ubiquitous so it's simply the most natural place to generate those snapshots (especially as CI runs tests, so it can automatically publish a new snapshot whenever tests pass).

If ~ in the prefix works, that would probably be easiest. But since you are GNU autotools compatible, I suspect it doesn't.

Gambit has the -:~~ runtime option to override installation directories; would that work?

[feeley]

Yes -:~~=... would work but will be painful. A slightly better option is to set GAMBOPT=~~=.... The best would be to have ~/.gambit/<hash> or whatever as a GAMBITDIR. Will check if this is possible.

[lassik]

The experience of following the snapshots was considerably different from running the latest release versions, as they constantly added new features/bugfixes and kept their main development branch quite stable so it was very rare for any features to be broken. This is the situation for Gambit as well. Would be interesting to find out how much demand there is for this thing :) Best case, it would attract new contributors to file bugs / PRs.

[lassik]

GAMBITDIR exists already in the repo but is not mentioned in the manual. Is it accessible to users yet?

[feeley]

GAMBITDIR is a build time symbol, not an environment variable. It is set by the configure script to the same value as prefix.

By the way, it seems to work when prefix is forced to ~/.gambit/foobar by adding an explicit assignment in the configure script. After a make, I get:

> (path-expand "~~")
"/Users/feeley/.gambit/foobar/"

But the make install failed to actually install in ~/.gambit/foobar for an unknown reason. But maybe adding the DESTDIR would solve that. I'll let you experiment... gtg...

[lassik]

Hmm. Did you manually change ./configure in a text editor? It's awesome that you got it to work that far but I'd start from the assumption that Autotools is "Jenga code" (https://en.wikipedia.org/wiki/Jenga) - one false move and everything falls down :)

[lassik]

Cirrus isn't picking up commits from the recreated udem-dlteam/gambit

[feeley]

Any progress on this front? It will be good to have a central CI system to build upon as libraries will also need to use CI, in particular the gambit/ffi/python library that is advancing well but needs tests on linux/macOS/Windows to verify that it works on the main platforms.

[feeley]

I just learned that Travis-CI now supports all 3 platforms, and that github has their own CI. Is there a good reason to consider Cirrus CI?

[daviwil]

Just an outsider with an opinion here :)

I've had a great experience using GitHub Actions to automate a cross-platform Gambit build for my own use:

https://github.com/daviwil/gambit/blob/build-gambit/.github/workflows/build-gambit.yml

If anyone is interested in using Actions I'd be happy to help answer questions or set something up. Nice thing about Actions is that it's easy to create a custom task that can be used across multiple repos, probably something that would be useful if Gambit's modules will be split into their own repositories.

[feeley]

Yes Actions seems like a good choice given it covers all majors platforms.

Could you look into replacing the current Travis-CI and Appveyor scripts?

[lassik]

@daviwil If you already have a solid understanding of Actions, trying it out sounds good.

[daviwil]

Sure! I'll put together a draft CI config that replaces what we've got for TravisCI and AppVeyor and send along a PR in a couple days.

[feeley]

Thanks!

[lassik]

Closing this issue since @daviwil has done good work on GitHub Actions based CI and there are no impediments in sight to adopting it wholesale for the main Gambit repo.