Open gilesknap opened 1 year ago
@coretl @ulrikpedersen tagging you both for a discussion of how to proceed with this.
It is possible that my native arm64 compile was done wrong in some fashion as it seems a very bad result.
I was wondering if the cause of the slow native build is to do with target. We are targetting linux:arm as that is what will be inside the ubuntu container running on a Mac. But the native compiler for Mac would be darwin-aarch64.
Would this maybe cause the epics build system to use cross compilation? The answer should be no as the build itself is running inside a linux container. Looking at the logs is looks to be targeting linux:arm and using gcc - I don't see any cross compilation.
It occurs to me that even if GitHub has real Macs for GHA they are probably not M1 as that is less than 3 years old. We are probably running on a x86 mac and getting QEMU to arm64 emulate for us!
doh!
@ulrikpedersen I'm merging into main all the changes except with arm64 approaches commented out. I can confirm that M1 is coming to GHA and will enable the native compile when this happens.
See https://github.com/github/roadmap/issues/528
The two approaches to building for arm64 are commented in .github/workflows/build.yml, see snippet below. The first approach is the fastest for now with build time 1h20m.
I can confirm that M1 is coming to GHA and will enable the native compile when this happens.
Very cool 👍 I was considering offering up my M1 Mini at home as a self-hosted runner. I even spent a couple of hours trying to figure out how - but I couldn't work out how to do it in a very safe way. It is not a dedicated machine for this task and has all my personal data on it.
In other news: I have bought a PC with Intel CPU and installed Ubuntu on it so that I can run both epics-containers and Tango containers without all the faff 😄 Looking forward to the day when GH releases Mac M1 runners though 🤞
Yeah self hosted runners are best done on isolated machines. We are planning to use machines in the IRIS cloud to do it.
FYI: as an esteemed customer of epics-containers I should warn you that Tom Gary and I have just sketched out a major refactor. The good news is that it should simplify the developer experience and align more closely with the standard devcontainer workflow. The bad news is its a bit of a rewrite of all the tutorials and I wont get started on that until mid Aug.
Thanks for the heads-up. I don't think it will affect us in how we are currently using epics-containers because we haven't fully adopted the workflow outlined in the tutorials...
@ulrikpedersen now is a good time to adopt the full epics-containers workflow if you were considering it.
Intro
I have successfully got arm64 builds of epics-base to work.
However there are a couple of reasons not to merge this into main.
Arm at DLS
Arm64 is useful for people using new macs for dev or running IOCs.
DLS's arm IOCs run on 32bit architectures armv7 and armv8 both of which have cross compilation in our current build system.
Even if we wanted to switch to doing native compile of these IOCs we could not as the 32 bit arm variants are lacking in package support and are not supported on GHA.
Hence these would be supported in containers as cross compile on linux:x86_64 just like RTEMS.
Arm64 Build Performance
I have tried two mechanisms for targeting arm64.
In both cases, this snippet in the Dockerfile instructs the epics build system of the target os:
The buildx approach uses QEMU to emulate arm64 for the compilation. This results in a build time of 1hr18m, see https://github.com/epics-containers/epics-base/actions/runs/5155081287
I had hoped that a native build on MacOS would improve things, but for whatever reason this is way worse at 3hr36m https://github.com/epics-containers/epics-base/actions/runs/5155705777.