Prepare for multi-arch Builds - ARM64

uselagoon / lagoon-images

This repository builds the images used in Lagoon and local development environments

Apache License 2.0

27 stars 32 forks source link

Prepare for multi-arch Builds - ARM64 #88

Closed dasrecht closed 2 years ago

dasrecht commented 3 years ago

As ARM64 architecture starts to get more common on desktops and the IaaS providers, we should also get ready to ship the lagoon images as multi-arch images.

I've started looking into building them on my arm64 node and will follow up with some inputs when I get the build to pass locally. But let's start the discussion of what would be needed (image-wise and when it comes to the build pipeline) to get the images shipped in our next releases.

ocean commented 3 years ago

So, it was a heck of an adventure, but I managed to get a functioning, minimalist stack of Lagoon (& GovCMS) containers running on my Apple M1 MBP.

I'll put in some PRs where I can, and am really wanting to contribute as much as possible back, but in summary quite a lot of it was rebuilding some of the many small tools (mostly Go ones) to run on linux/arm64, and then using Docker Moby Buildkit to build linux/arm64 lower level images to base my containers off.

I've put a PR in for cross-compiling ahoy, and will put in more for docker-gen, envplate, dockerize, trivy, pygmy and the docker-haproxy image once I get proper multi-arch builds going for them.

tobybellwood commented 3 years ago

bumpyride

dasrecht commented 3 years ago

@ocean I was able to get around trivy by using - jitesoft/trivy instead - https://github.com/aquasecurity/trivy/issues/496

Beware this might a we bit dated but my findings:

ES6 and Kibana6 - I've dropped from my tests as it won't build - but I think we'll EOL them in one of the next releases.
Elasticsearch 7.6.1 hasn't been built for ARM but after 7.8.1 there are base images.
Tini (if not installed via apk) needs to be adapted to download the correct architecture version - https://github.com/krallin/tini/releases (seen here : https://github.com/uselagoon/lagoon-images/blob/main/images/elasticsearch/7.Dockerfile#L17)

I'll give it a rerun over the weekend and see where we land with the current main branch :)

Apart from that.. building on the SDCard of my ARM64 box might not be the wisest decision 🙈

ocean commented 3 years ago

@dasrecht Nice find with the alternate Trivy build. After following that I've put in a PR for Trivy to build multi-platform Docker containers, so hopefully that will be merged soon and then the original repo can be used again.

(Aside: check out the goreleaser tool that they (and loads of other Go apps) use to build and publish binaries and images, it's pretty amazing!)

I'm still running an older version of these images for my local stack (from the 1.6.0 branch of amazeeio/lagoon, as that's where our stack is currently at 😞 ) but I'll get onto building and running the latest set of images from this repo soon on my M1 as we upgrade our stack.

Eek, nasty load average 🙈 Mem use is about the same for me, except it's in the 4 GB I've allocated to my Docker Desktop, heh.

ocean commented 3 years ago

@dasrecht @tobybellwood I've just added a small PR to get discussion going https://github.com/uselagoon/lagoon-images/pull/150

ocean commented 3 years ago

@dasrecht As of Trivy version 0.17.x (they're up to 0.17.2 now with a couple of extra fixes), their Docker images are multi-platform (since they merged my PR 🎉 😁 ).