unable to install when building docker image on Apple Silicon `arm64` without `buildx`

[98sean98]

The meteorite package doesn't seem to be installable within a docker image build for arm64.

Example dockerfile:

FROM python:3.11-slim-buster

RUN pip install torch torchvision torchaudio meteorite

An error is thrown during meteorite installation with the following:

ERROR: Could not find a version that satisfies the requirement meteorite (from versions: none)
ERROR: No matching distribution found for meteorite

But it is installable if buildx build --platform linux/amd64 is used.

However, I've been able to install meteorite in a normal python environment on Apple Silicon arm64 with the meteorite-0.0.5-cp310-cp310-macosx_10_9_x86_64.macosx_11_0_arm64.macosx_10_9_universal2.whl wheel for python 3.10.

[utkarsh867]

It seems like the issue is from the absence of processor information/different machine type as read by Python

This is what I get in Docker build

This is what I get when I run it natively:

The solution at the moment would be to use BuildKit to install.

[utkarsh867]

The issue however, seems purely because of the suffix of the wheel.

For example, running a docker buildx build --platform linux/arm64 yields the same results, except for the machine, and installs the pip package successfully

Interestingly, running a docker buildx build --platform linux/arm64 has no effect on the machine name, which tells me that Docker follows this convention, and we might need to build wheels with aarch64 suffix as well.

I think it is just the convention that arm linux uses, with a aarch64 suffix.

[utkarsh867]

And finally, we don't yet support arm on linux yet. We can support ARM on macOS because it is a universal2 binary. Unfortunately, this has to do with the unavailability of ARM runners on Actions.

Cross-compilation could resolve the issue, but comes with its own complexity. I did explore that route previously, but I was mostly unsuccessful in the amount of time I spent on it.

[98sean98]

I understand the problem now. The lack of built wheel distribution specifically for linux arm64 or aarch64 is causing pip to throw ERROR: No matching distribution found for meteorite. And as you've mentioned, the lack of ARM runners on Github Actions is a bottleneck, unless we choose to use 3rd party or self-hosted runners.

To be very frank, is there a lot of value in supporting linux arm right now? If not, I don't see why we have to build for this right now.

[utkarsh867]

I think we can leave it out for now.