Add arm64 image build and switch build to use Docker BuildX

kaitlinsm commented 3 years ago

Checklist

[x] Implementation
[ ] Tests
[ ] Docs

Implementation:

Switched build system to use Docker BuildX, allowing targeting of multi-arch docker images. Currently only building arm64 and amd64 arch images, but it might be good to add other ARM architectures if anyone wants to run containerised on an older Raspberry Pi.

Tests:

Build process tested against my own Docker repo. I've assumed that the repo secrets DOCKER_PASSWORD and DOCKER_USERNAME are the login details for the inspircd/inspircd-docker Docker Hub repo, and I've made sure that the build works pushing to my Docker Hub. I've verified that the INSP_VERSION env var is respected and added to the BuildX args, so it definitely pulls the correct tagged version of the inspircd source. Can't really test this bit as it's the GitHub actions build script...

Docs:

Don't think any explicit docs are needed. My testing on my own multi-arch k8s cluster shows that the images can be happily used on whatever host, and image pulls will select the correct architecture for the host automatically.

Caveats:

Build time of the GitHub build action is quite long, increasing from ~10mins to ~50mins. I've run a test build on my Raspbery Pi 4, and the build time on that system is not far off that upper ballpark. I've had a look at the caching of the docker/build-publish step, and I don't think it's likely to help much seeing as this image only gets built once in a while.

Merging:

If this PR is merged, then the build action will kick off, and it will REPLACE all currently tagged images in the docker hub repo - so the current amd64 images tagged 3/3.10/3.10.0/latest will be replaced. Not sure it'll be a problem, but just something to be aware of.

kaitlinsm commented 3 years ago

So the PR build failed - it's failing to log in to docker hub. The build script uses the following:

        name: Login to DockerHub
        uses: docker/login-action@v1 
        with:
          username: ${{ secrets.DOCKER_USERNAME }}
          password: ${{ secrets.DOCKER_PASSWORD }}

Which worked in my fork, because I had two secrets called DOCKER_USERNAME and DOCKER_PASSWORD. I've removed setting the secrets in the env block, as some build actions echo out the entire env block to their log when they run, which is obviously bad. But I can't see if I've got the right secrets. I thought I had but... yeah. I'm a bit baffled as to why this is failing. I could disable the docker login step if it's not a master build, I suppose?

kaitlinsm commented 3 years ago

Disabling the docker hub login step unless it's a master build seems to make sense to me - there's no need to log into a service that we're not going to use.

I'm at the end of my rope now - since I have no visibility of this repo's secrets, I can't make sure that DOCKER_USERNAME and DOCKER_PASSWORD will work when used by the docker/login-action@v1 step.

kaitlinsm commented 3 years ago

Some tests fail when run locally. After doing docker swarm init and docker build -t inspircd:testing ., the following four tests fail:

customConfigSecrets.sh:, secrets.sh

image inspircd:testing could not be accessed on a registry to record its digest. Each node will access inspircd:testing independently, possibly leading to different nodes running different versions of the image.

default.sh:, mountConfig.sh

SSL handshake has read 2602 bytes and written 709 bytes
Verification error: unable to verify the first certificate

All other tests pass.

The certificate issues, I don't really get.

The other issue - "image could not be accessed on a registry" - could have a work around. Had a look at this: https://codeblog.dotsandbrackets.com/private-registry-swarm/ and the suggestion is to create a service that runs a registry and force docker to use and trust that registry, then build and publish the image to that registry in the test context, and then run the tests. Unless I've misunderstood this whole process.

Other observations:

The Docker BuildX step doesn't leave the built image lying around
we want to build and test images before merging.
in order for the tests to pass, it looks like docker needs to run an image available in an image repository (??? might have misunderstood this!)
we don't want to leave test images lying around in a public repo, so we don't want to push them to Docker Hub
we could use GitHub Container Registry for test images, setting up a separate build step just for that repo, and delete the image from the repo after testing is finished - see https://github.com/docker/build-push-action/blob/master/docs/advanced/isolated-builders.md and https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-container-registry )
this would only work for builds against the main repo, not for forked repos (the auth secrets necessary to login wouldn't be present on forks)

the workflow i'm proposing is:

if building a PR:

modify run_tests.sh to allow overriding of the test image name. locally, it can still be (and defaults to) inspircd:testing Option A:
build amd64 image only
push it to GHCR as inspircd:testing-$SOME_BUILD_IDENTIFIER
execute run-tests.sh with that image - e.g. export $TEST_IMAGE=gchr.io/inspircd/inspircd:testing-$SOME_BUILD_IDENTIFIER
once test passes/fails, delete the image (<- not sure how to do this!) Option B:
run a registry service
force docker to use and trust that registry
build the docker image tagged as inspircd:testing and then push to transient registry service

if building a merge to master

don't run the tests, because managing the test images is a pain. If only PRs where the tests pass are merged, then the tests will have been run in the GitHub action when the PR was being built.

I could be a bit wrong though.

inspircd / docker