search

epics-containers / epics-base

A container image for EPICS base libraries and tools built for linux-x86_64 and RTEMS
Apache License 2.0
6 stars 4 forks source link

Adding support for Arm64 #12

Open gilesknap opened 1 year ago

gilesknap commented 1 year ago

Intro

I have successfully got arm64 builds of epics-base to work.

However there are a couple of reasons not to merge this into main.

Arm at DLS

Arm64 is useful for people using new macs for dev or running IOCs.

DLS's arm IOCs run on 32bit architectures armv7 and armv8 both of which have cross compilation in our current build system.

Even if we wanted to switch to doing native compile of these IOCs we could not as the 32 bit arm variants are lacking in package support and are not supported on GHA.

Hence these would be supported in containers as cross compile on linux:x86_64 just like RTEMS.

Arm64 Build Performance

I have tried two mechanisms for targeting arm64.

In both cases, this snippet in the Dockerfile instructs the epics build system of the target os:

FROM base AS environment-amd64
ENV EPICS_HOST_ARCH=linux-x86_64

FROM base AS environment-arm64
ENV EPICS_HOST_ARCH=linux-arm

FROM environment-$TARGETARCH AS environment

The buildx approach uses QEMU to emulate arm64 for the compilation. This results in a build time of 1hr18m, see https://github.com/epics-containers/epics-base/actions/runs/5155081287

I had hoped that a native build on MacOS would improve things, but for whatever reason this is way worse at 3hr36m https://github.com/epics-containers/epics-base/actions/runs/5155705777.

gilesknap commented 1 year ago

@coretl @ulrikpedersen tagging you both for a discussion of how to proceed with this.

It is possible that my native arm64 compile was done wrong in some fashion as it seems a very bad result.

gilesknap commented 1 year ago

I was wondering if the cause of the slow native build is to do with target. We are targetting linux:arm as that is what will be inside the ubuntu container running on a Mac. But the native compiler for Mac would be darwin-aarch64.

Would this maybe cause the epics build system to use cross compilation? The answer should be no as the build itself is running inside a linux container. Looking at the logs is looks to be targeting linux:arm and using gcc - I don't see any cross compilation.

gilesknap commented 1 year ago

It occurs to me that even if GitHub has real Macs for GHA they are probably not M1 as that is less than 3 years old. We are probably running on a x86 mac and getting QEMU to arm64 emulate for us!

doh!

gilesknap commented 1 year ago

@ulrikpedersen I'm merging into main all the changes except with arm64 approaches commented out. I can confirm that M1 is coming to GHA and will enable the native compile when this happens.

See https://github.com/github/roadmap/issues/528

The two approaches to building for arm64 are commented in .github/workflows/build.yml, see snippet below. The first approach is the fastest for now with build time 1h20m.

https://github.com/epics-containers/epics-base/blob/96f1abad791614caa03b8cd9bae22b36f083d79b/.github/workflows/buiild.yml#L7-L41

ulrikpedersen commented 1 year ago

I can confirm that M1 is coming to GHA and will enable the native compile when this happens.

Very cool 👍 I was considering offering up my M1 Mini at home as a self-hosted runner. I even spent a couple of hours trying to figure out how - but I couldn't work out how to do it in a very safe way. It is not a dedicated machine for this task and has all my personal data on it.

In other news: I have bought a PC with Intel CPU and installed Ubuntu on it so that I can run both epics-containers and Tango containers without all the faff 😄 Looking forward to the day when GH releases Mac M1 runners though 🤞

gilesknap commented 1 year ago

Yeah self hosted runners are best done on isolated machines. We are planning to use machines in the IRIS cloud to do it.

FYI: as an esteemed customer of epics-containers I should warn you that Tom Gary and I have just sketched out a major refactor. The good news is that it should simplify the developer experience and align more closely with the standard devcontainer workflow. The bad news is its a bit of a rewrite of all the tutorials and I wont get started on that until mid Aug.

ulrikpedersen commented 1 year ago

Thanks for the heads-up. I don't think it will affect us in how we are currently using epics-containers because we haven't fully adopted the workflow outlined in the tutorials...

gilesknap commented 6 months ago

@ulrikpedersen now is a good time to adopt the full epics-containers workflow if you were considering it.