Convert Linux/macOS Travis CI to use GitHub Actions

[ronaldtse]

Description

GitHub Actions is now available for the rnpgp account, and we should utilize it. The performance provided is notably quicker than Travis.

[ronaldtse]

@dewyatt @ni4 GitHub Actions are way faster than Travis, and also allows more concurrent builds. We should move over ASAP (it also allows Windows testing, so we no longer need AppVeyor).

[ronaldtse]

@dewyatt could we prioritize this issue? It's going to bring down the run time by quite a bit. Thanks!

[ronaldtse]

@dewyatt are you already working on this? Otherwise, perhaps @ni4 could help...

[dewyatt]

@ronaldtse I have indeed started on it, I'm just switching back and forth between this and enprot things right now.

[ronaldtse]

@dewyatt then perhaps @ni4 should start on a separate effort? Or should he help with GHA on Windows?

[dewyatt]

@ronaldtse Either way works for me. I haven't touched the Windows side of things, so that's certainly an option.

[ronaldtse]

@dewyatt then let's do that! @ni4 could you help with setting up GHA for Windows?

This task will now only be about macOS and Linux. Windows is handled in #892 .

[dewyatt]

Just an update, I've been working on this but it will take me a few weeks I think. I know it seems simple on the surface but there are various tricky issues. Most of the issues I'm encountering are ones I've already solved in my gitlab-ci work that I never quite finished, so that helps speed things along. But some are a result of newer changes in the build, etc.

Right now I'm looking at building base docker images with all deps since GHA doesn't really have any caching yet.

[ronaldtse]

@dewyatt no problem. I think it would be worthwhile getting GHA working incrementally (even without Docker) due to the parallel builds GHA does. For the build images, we can use GitHub packages to store the Docker containers.

[dewyatt]

@ronaldtse Well Travis does parallel builds too, but also has a caching mechanism (botan takes a while to build, for example), so I'm not expecting much improvement without doing the docker images. And yes I'm planning on trying GitHub Package Registry for the images.

[dewyatt]

I guess having said that, we could try it without the caching/images first if you'd prefer. I suppose I could be prematurely optimizing. :D

[ronaldtse]

@dewyatt Travis only allows parallelizing three builds at once for open source projects. GHA does the builds all at once. For Metanorma, our build time (admittedly it's a stack) has reduced from nearly a day to within an hour 😜

[dewyatt]

@ronaldtse I think travis does 5 at once and we only have 7 here. Those are some amazing improvements on Metanorma though, wow!

EDIT: Huh maybe travis does do 3, I could have sworn it did 5 in the past?

[ronaldtse]

@dewyatt I'm not sure why but Travis did become slower in terms of parallelization, it might be because we used heavier matrices... In any case iterative improvements are good. Might already be faster than Travis 😄

[dewyatt]

This may not be as bad as I thought. I got a basic matrix mostly working today. cli_tests SignDefault suite is failing for some reason, so I'll have to look into that. The sanitize build isn't working but I believe I know what needs to be done there from the gitlab-ci work.

I think the normal run was about 30m so it's slower than travis, probably will need the docker images but that's not difficult. GHA lets you do cron scheduled jobs so I can probably use that to occasionally build dev images.

Anyway, hopefully this coming week (maybe next) I'll at least have Linux transitioned over.

By the way it looks to me like travis-ci.org did/does 5 concurrent jobs for OSS, travis-ci.com does 3. (that explains my confusion I guess)

[ronaldtse]

@dewyatt this is why we dread having to wait for Travis...

[dewyatt]

@ronaldtse Yep I hear you on that!

As far as the Linux side of things here, I think I only have one issue to track down in cli_tests in order to get things passing in centos:7. So I'll be looking into that and then working on the macOS side more (currently for macOS I'm just getting "This check failed" with no information).

[ronaldtse]

@dewyatt that's great. Can you make a draft PR for GHA? For macOS it might be because GHA changed the macOS environment to macos-latest which runs Catalina and Xcode 11.1. Their email says that specifying other values will cause the macOS builds to fail.

[dewyatt]

@ronaldtse Sure I'll get a draft PR going. Yep changing to macOS-latest makes things at least try to work, so that's an improvement!

[ni4]

@dewyatt @ronaldtse Should we close this since GHA is up and running?

[ronaldtse]

Yes please. Thanks!