Migrate push build pipelines from Drone to GitHub Actions

[r0mant]

Drone pipelines

Drone currently runs the following pipelines on every push to master and branch/v* release branches:

• [x] update-docs-webhook
• [x] push-build-linux-amd64
• [x] push-build-linux-386
• [x] push-build-linux-amd64-fips
• [x] push-build-windows-amd64
• [x] push-build-darwin-amd64
• [x] push-build-native-windows-amd64
• [x] push-build-linux-arm
• [x] push-build-linux-arm64
• [x] build-buildboxes

update-docs-webhook triggers docs deployment in Vercel, build-buildboxes builds and pushes buildbox images to ECR, other push-* pipelines build Teleport artifacts on different architectures.

Scope

Let's start with push-* pipelines and then do buildboxes and docs.

Notes

These pipelines live in .drone.yaml, keep in mind the file (at least all push-* pipelines) is generated by dronegen:

https://github.com/gravitational/teleport/blob/362cb4641281f97e417e3e348958ded31e4d7108/.drone.yml#L38

Some other implementation notes:

• The pipelines clone teleport.e repository so GHA workflows should probably live in teleport.e repository.
  • The buildboxes pipeline will require permissions to push to ECR, so another reason to keep them in enterprise.
• arm64 pipeline already runs on GHA, Drone just triggers it via API. We can use the same approach for graceful migration.
  • https://github.com/gravitational/teleport.e/pull/673

[camscale]

Some work has been done on changing the buildboxes to be based on centos7 and to run on a native-ish architecture, which provides the following benefits:

• The native-ish build (arm64 runners to build arm64 and arm [32-bit], amd64 runners to build amd64 and 386 [32-bit]) makes it easier to make fully featured builds which include PIV and BPF support. As these are implemented by C libraries, they need a toolchain to be able to build these features. Cross toolchains for C is quite a bit tricker than Go, so it is more easily done by having a native toolchain rather than a cross build toolchain).
• Centos7 is the oldest target we support for Linux. Building on that means we will not have glibc compatibility issues which can arise when building against a later glibc and deploying on an earlier one such as centos 7. Building on the earliest supported glibc should be fine for later releases of glibc on more modern distros.

This work has been started in:

• https://github.com/gravitational/teleport.e/pull/890
• https://github.com/gravitational/teleport.e/pull/926

and likely other PRs.

One issue with building on Centos 7 is it does not support node.js later than v16 and we need to use a later version for more modern Teleport Connect builds. https://github.com/gravitational/teleport/issues/21724 tracks this. The plan is to build the teleport binaries with a Centos 7 buildbox and take the tsh binary from that for Teleport Connect, which can be built on a more modern buildbox. This will forego running Teleport Connect on Centos 7, but that does not seem like it will be an issue as that is a server-based OS and not likely used a desktop OS, and Teleport Connect is a desktop app.

For the purposes of migrating the build pipelines off drone to GHA, we could leverage the work to use centos7-based buildboxes, but there is still some more work to be done there - at the very least, the arm64 buildbox does not appear to have the arm (32-bit) toolchain available so the build for arm fails with that.

As a first step to move off drone, we'll keep the existing buildboxes as used by drone and implemented in the build.assets/Makefile build targets. This is a fairly simple migration to get off drone. Then we can rev that to move the builds to be centos7-based and split off the Teleport Connect build to a different buildbox with a more modern version of node.js.

The current dispatchable workflow for linux builds (release-linux-arm64.yml) is not suitable for broader integration with drone in its current form:

• It is ARM64-specific,
• It calls build-linux-generic.yml which is based around the centos7 buildboxes
• It uses a GHA matrix build - while suitable for a final non-drone solution for running the full suite of release builds, it does not integrate well with the individual drone pipelines calling GHA to build each separate release.

This this in mind, the following steps will be taken:

• Create a new dispatchable GHA workflow for drone to call to build using the existing buildboxes
• Create a new callable GHA workflow to call the release targets in build.assets/Makefile
• Add an optional step to create a Teleport Connect release to the previous workflow, to be used only for the amd64 release

Once that is done, #20729 can progress with the tag build pipelines using the same workflows, just with an additional parameter specifying that artifacts be released as is currently done with the ARM64 and MacOS release builds.

This will take care of most of the push build pipelines leaving:

• build-buildboxes
• update-docs-webhook
• push-build-native-windows-amd64

The bulld-buildboxes workflow is a similar migration to the push build above - it is just calling targets in build.assets/Makefile and pushing the resulting images to an ECR registry.

The update-docs-webhook appears trivially implementable with a curl command in a github action. More details of the webhook are still to be understood

push-build-native-windows-arm64 is still an unknown. Investigation into Windows builders on GHA still to be done.

[fheinecke]

@camscale do you think that this is good to close out, or do you want to wait until after Drone no longer drives/calls the actions?

[zmb3]

@camscale @fheinecke I assume this is good to close now?

[camscale]

Not quite. We have disabled push builds for now because we're not sure how to trigger them well from the OSS repo. I am doing a workflow cleanup and optimisation this week and next and I'll be looking at what options we might have available then. The alternative, depending on how fast I can get the release build process, is to have a release build each merge, pushed to the stage environment.

Alternatively we can close this and I can open a new issue for what might do.

[camscale]

On second thoughts, lets close this. We have migrated everything from drone. The rest is a change in the way we do things.