Build the images for arm allowing for native Apple Silicon support

[liadmord]

Mac machines using Apple Silicon become more and more popular, currently in order to run. docker-osx images on such machines requires using the --platform linux/amd64 flag which makes the container slower and prevents using binaries built on the host machine.

This can be easily solvable by having a built version for arm for all the images.

[maltegrosse]

a multi-arch build would be very useful (arm64v8)

• i am just not sure if arm64v8 macOS images are available

[sickcodes]

I can try with some of these:

https://hub.docker.com/search?q=archlinux&type=image&architecture=arm64

Looks like these could help https://hub.docker.com/r/sapk/archlinux

docker build --build-arg ARCH="arm" --build-arg QEMU_ARCH="arm" --build-arg REPO="http://mirror.archlinuxarm.org/arm/\$repo" -f Dockerfile.multi -t sapk/archlinux:arm .
docker build --build-arg ARCH="aarch64" --build-arg QEMU_ARCH="aarch64" --build-arg REPO="http://mirror.archlinuxarm.org/aarch64/\$repo"  -f Dockerfile.multi -t sapk/archlinux:aarch64 .
docker build --build-arg ARCH="armv6h" --build-arg QEMU_ARCH="arm" --build-arg REPO="http://mirror.archlinuxarm.org/armv6h/\$repo"  -f Dockerfile.multi -t sapk/archlinux:armv6h .
docker build --build-arg ARCH="armv7h" --build-arg QEMU_ARCH="arm" --build-arg REPO="http://mirror.archlinuxarm.org/armv7h/\$repo"  -f Dockerfile.multi -t sapk/archlinux:armv7h .
docker build --build-arg ARCH="x86_64" --build-arg QEMU_ARCH="x86_64" --build-arg REPO="https://mirrors.kernel.org/archlinux/\$repo/os/x86_64"  -f Dockerfile.multi -t sapk/archlinux:x86_64 .
docker build --build-arg ARCH="i686" --build-arg QEMU_ARCH="i386" --build-arg REPO="https://mirror.archlinux32.org/i686/\$repo"  -f Dockerfile.multi -t sapk/archlinux:i686 .

Well we have iOS running here: https://hub.docker.com/r/sickcodes/docker-eyeos

So it's probably possible.

I don't have an M1 to test on

Apple added M1 support themselves: https://developer.apple.com/documentation/virtualization/installing_macos_on_a_virtual_machine

Does that help?

[sickcodes]

https://hub.docker.com/u/arm64v8/

arm64v8 docker images are available, so it's definitely possible

[maltegrosse]

Yes, armv64v8 images are available. But I am abit worried about the main macOS dmg file as you can’t choose the architecture See https://github.com/kholia/OSX-KVM/blob/master/fetch-macOS-v2.py#L119

[maltegrosse]

@sickcodes I started a bit playing around with your docker images. As I am not very familiar with arch linux, I struggled with x64 only packages in the aur repository. Before rebuilding packages, I decided to port your Dockerfiles to Debian. I ve created the base (perhaps renaming the 'latest' to 'base' tag would make sense?), the naked and the auto image for armv8 and x86. I currently dont have access to my kvm enabled machine - so I cant test the results. Can you give a try if the x64 images are still working for you? https://hub.docker.com/r/maltegrosse/docker-osx (the images seem to be smaller, less ~500mb)

if working - I would consider some cleanup and later a PR

Update 2022-03-06a: Upgraded Debian Buster to Bullseye to have a newer QEMU version available. Update 2022-03-06b: on native hardware: armv8 image is booting (ends up in kernel panic), but as assumed earlier, the osx (catalina) image runs as x86/64 only (Kernel Version ..... xnu-6153.141.2.2~/RELEASE_X86_64) - qemu shows a x64 process on arm64 device. Update 2022-03-06c: Once went over this article, I figured out that apple provides universal binary images since big sur (for arm64 and x86). As my current kernel panic is about the wrong cpu information, can someone provide me the correct a1 information (see cpu and cpuid)

[sickcodes]

I believe it's 100% possible. I unfortunately don't have an M1 to test on.

I assume M1 should be dirt cheap soon, so I could assist.

Yeah so the blackberry article uses OSX-KVM which is what we use.

Also, regarding the Arch instead of Ubuntu, is that I like to keep this project cutting edge and that includes fixing bugs or raising upstream issues so we can truly contribute :)

[sickcodes]

If you'd like, I would be totally open to docker-osx:catalina-buster if you want to PR them back as Dockerfile.debian :)

[maltegrosse]

I tried both images on my x86 machine, my docker image based on Debian ends up in kernel panic -so there is still alot of work todo. Anyway, I decided not to continue building these images, because even if the docker image is made for arm64, it still emulates qemu for x86. see qemu x86

To achieve a native arm64 support, work from this article needs to be adopted. (see their github) And even after months with a lot of expertise in this topic (which I dont have) - this team "just" got the kernel running.

A multi-arch image could be made with their qemu version - but beside running the kernel there wont be a lot to see. It took years to somehow emulate stable macos on non native x86 apple hardware - so the same for arm64... arm macos on arm mac hardware would be the first case, once this is done, it could be ported to other arm hardware. (rpi etc)

[ylluminate]

Just want to be sure I'm interpreting the purpose of this issue clearly: The goal here is to be able to run x64 + Apple Silicon (ARM64) versions of Linux AND macOS via Docker-OSX on Apple Silicon (M1+) based Apple computers, right?

[liadmord]

I think that the most relevant case here would be to run an ARM64 macos container on m1 based computers, I also think that the challenge here is not running containers but creating them, once created running them would be easy.

[maltegrosse]

I think that the most relevant case here would be to run an ARM64 macos container on m1 based computers, I also think that the challenge here is not running containers but creating them, once created running them would be easy.

The challenge is not the container, the challenge is the qemu including all the customization to run arm macos (see my comment above) If OSX-KVM has native arm64 support, the next step would be containerization-so this issue should addressed there

[joshuataylor]

:wave: Thanks so much for this fantastic repository and the research notes above.

I've started digging into this, and have also experienced the same with Archlinux and aur being mostly amd64 and not ARM. So we'll need to build from source until this is fixed (which is fine, we could even contribute back upstream once this is running by contributing the packages).

I'm testing both on an actual M1 Macbook Air 2020 and two linux hosts, one running Archlinux and one running Ubuntu 22.04. Both hosts SHOULD work with minor tweaks, which I plan to document (you basically need to run docker as aarch64 so it builds).

We don't have many options for running Archlinux yet, so right now I'm using lopsided/archlinux-arm64v8 which has a daily build of archlinux then installing base-devel on top of that. Not a major issue.

My primary goal is to use this to build packages for m1 CPUs (oh this rabbithole 💯), so GPU acceleration, USB passthrough etc won't be my priority but the work from this thread has laid some excellent groundwork.

[joshuataylor]

So after digging into this more, it's becoming more and more of a rabbithole.

I have learnt a lot about how everything ties together, 24 hours ago I had 0 experience with the Mac side of things (coming from a Linux background).

Comments: 0/ If you just want Mac M1 on Mac M1, this could be achieved using the apple virtualisation work they've been doing, I think. I don't want this, my goal is to build packages via an Apple M1 system. 1/ Asahi Linux is making great strides with things such as their [generic bootloader])(https://github.com/AsahiLinux/m1n1), which could be of use. Plus their wiki/research notes are great. 2/ The OVMF is compiled through KVM-Opencore, which relies on a few libraries to build which can't be built using arm64/aarch64, such as Lilu -- thread here https://github.com/acidanthera/bugtracker/issues/1928 3/ The notes in various repos are great. These are for x86_64, which is not what we want but an incredibly helpful starting point for learning about the Mac ecosystem. 4/ We'll need to build various things to make this work, but a lot of the work is already there. QEMU is flying along with all their updates at breakneck speed, the work around bootloaders by the existing projects is great as well. Many of the projects already have the initial ARM64/AARCH64/ARM64e/WHY CAN'T THIS JUST BE NAMED SOMETHING STANDARD.

[ylluminate]

https://github.com/sickcodes/Docker-OSX/issues/547