Wheel support for linux aarch64

[odidev]

Summary Installing mypy on aarch64 via pip using command "pip3 install mypy" tries to build wheel from source code.

Problem description mypy doesn't have wheel for aarch64 on PyPI repository. So, while installing mypy via pip on aarch64, pip builds wheel for same resulting in it takes more time to install mypy. Making wheel available for aarch64 will benefit aarch64 users by minimizing mypy installation time.

Expected Output Pip should be able to download mypy wheel from PyPI repository rather than building it from source code.

@mypy-team, please let me know if I can help you building wheel/uploading to PyPI repository. I am curious to make mypy wheel available for aarch64. It will be a great opportunity for me to work with you.

.

[ethanhs]

This issue should go to https://github.com/mypyc/mypy_mypyc-wheels. I've opened https://github.com/mypyc/mypy_mypyc-wheels/issues/12

[hauntsaninja]

Note mypy ships with a pure Python wheel, so you shouldn't be seeing anything building... (also mypy_mypyc-wheels has a PR that makes a large number of changes pending, so I'd hold off on making a PR till that gets merged)

[JukkaL]

Let's keep the issue also open here for more visibility, since not many users know about https://github.com/mypyc/mypy_mypyc-wheels.

[uSpike]

I've opened https://github.com/mypyc/mypy_mypyc-wheels/pull/37 which adds this support and also arm64 on MacOS. Would appreciate a workflow approval, review, and test on a Mac since I don't have one :(

[mr-c]

https://github.com/mypyc/mypy_mypyc-wheels/pull/37 was closed as the build took too long. Would a PR to build linux aarch64 wheels natively on circleci (for free) similar to https://github.com/zeromq/pyzmq/pull/1743 be accepted?

[aiven-anton]

Is it possible to give some feedback to the above to unlock this situation, or is this issue essentially blocked due to no good enough solutions for building wheels?

[isuro]

Another bump on this, specifically on the question of whether a PR containing a CircleCI config would be desirable. Thanks! (@hauntsaninja?)

[hauntsaninja]

I don't think I have perms to hook up https://github.com/mypyc/mypy_mypyc-wheels to a new CI provider and if it breaks I probably won't have time to fix it. But those caveats aside, seems like a reasonable thing to make a PR for

[jriddy]

As mentioned here, it's probably possible to emulate arm64 builds on a macos gha runner with colima/docker. With no need for CPU emulation, the build should be performant enough.

I'm trying to find the time to do this.

[achimnol]

~~https://github.com/lablup/backend.ai-oven/tree/main/pypi/projects/mypy I'm building these by myself (using OrbStack on my Apple Silicon laptop) and want to see the official builds soon.~~ Ah, @jriddy already mentioned my previous comment. (I just forgot that I've already left it... 🥲)

The build steps are fairly easy enough (see build.md there), but it seems that it takes too much time to bootstrap all the compiler toolchains such as LLVM. It takes about 2 hours in M1 Max. If we could use some prebuilt container images with the toolchain already installed, we could reduce the build time more.

As of 2023 October, GitHub Actions has Apple Silicon runners. If we could utilize them, we could meet the hard time limit of the action workflow (the 6 hours limit as mentioned) to build the linux aarch64 mypyc-optimized wheels.

[jriddy]

That shouldn’t be too hard to do

On Tue, Dec 26, 2023 at 01:30 Joongi Kim @.***> wrote:

https://github.com/lablup/backend.ai-oven/tree/main/pypi/projects/mypy I'm building these by myself (using OrbStack https://orbstack.dev on my Apple Silicon laptop) and want to see the official builds soon.

The build steps are fairly easy enough, but it seems that it takes too much time to bootstrap all the compiler toolchains such as LLVM. If we could use some prebuilt container images with the toolchain already installed, we could reduce the build time more.

— Reply to this email directly, view it on GitHub https://github.com/python/mypy/issues/9688#issuecomment-1869295924, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAH5UH5KSOLWDJ4MKA5OEDTYLJVHLAVCNFSM4THG7HL2U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCOBWHEZDSNJZGI2A . You are receiving this because you commented.Message ID: @.***>

[achimnol]

@hauntsaninja Just searched a little bit about this, but unfortunately........ https://docs.github.com/en/actions/using-github-hosted-runners/about-larger-runners/about-larger-runners#limitations-for-macos-larger-runners

Nested-virtualization is not supported by arm64 runners, due to a limitation of Apple's Virtualization Framework that our hypervisor uses.

😭 🫠

[achimnol]

I have heard some of best testing applicants for Linux arm64 runners got access. Maybe we should wait for GitHub to make it generally available...

[jriddy]

@hauntsaninja Just searched a little bit about this, but unfortunately........ https://docs.github.com/en/actions/using-github-hosted-runners/about-larger-runners/about-larger-runners#limitations-for-macos-larger-runners

Nested-virtualization is not supported by arm64 runners, due to a limitation of Apple's Virtualization Framework that our hypervisor uses.

😭 🫠

I was thinking this (pre-release) action might work to allow running linux containers via colima (which i'm going to assume interacts with the hypervisor in some way to get its linux VM rather than being "nested"). Although given that it's an "alpha" release I have no idea how viable this is for real workloads.

[achimnol]

https://github.blog/2024-06-03-arm64-on-github-actions-powering-faster-more-efficient-build-systems/ Now we have it!

[JukkaL]

Does anybody know the status of GitHub arm support? It would be nice to have arm wheels available for Linux. The blog post referred to above said that these should be available for open source projects by the end of the year:

These runners are available to our customers on our GitHub Team and Enterprise Cloud plans. We expect to begin offering Arm runners for open source projects by the end of the year.