search

python-cffi / cffi

A Foreign Function Interface package for calling C libraries from Python.
https://cffi.readthedocs.io/en/latest/
Other
108 stars 40 forks source link

Please provide Windows ARM64 wheels #109

Open nanonyme opened 1 month ago

nanonyme commented 1 month ago

There is now native ARM64 Python since release 3.12. many central projects including this one do not provide wheels for this platform. Please start building wheels also for Windows ARM64 when convenient.

ofek commented 3 weeks ago

Yes this would be fantastic, we've been getting issues where users can no longer install our packages.

nanonyme commented 3 weeks ago

It's possible this is pending GitHub ARM64 Windows project like for many other Python projects. That in term is most likely pending Windows Server 2025 release which is the first release to support ARM64.

ofek commented 3 weeks ago

Cross compilation is possible, for example cibuildwheel already supports doing that for Windows ARM.

mattip commented 3 weeks ago

Native Arm64 windows github action runners are in beta. I am not aware of an easy way to get docker or qemu to run windows-arm64 on a github CI runner for testing.

users can no longer install our packages

You imply that once they could and now they can't. Is that what you meant?

Cross compilation is possible

Yes, but that does not solve the testing problem. I would be hesitant to release a package like CFFI without extensively testing that it works on a new platform. This is also the approach NumPy has taken: it will not release windows-arm64 wheels without testing. Here is the relevant comment chain on the issue.

mattip commented 3 weeks ago

A quick search on another base package in the python extensions multiverse, cryptography, resulted in a similar approach: no wheels without testing.

nanonyme commented 3 weeks ago

Native Arm64 windows github action runners are in beta. I am not aware of an easy way to get docker or qemu to run windows-arm64 on a github CI runner for testing.

Yeah, I've been digging around and the reason it's only in beta even though Microsoft is heavily pushing ARM64 most likely is because there is no existing non-beta Windows Server that can run on ARM64. It's no doubt going to be released any year now.

nanonyme commented 3 weeks ago

Yes, but that does not solve the testing problem. I would be hesitant to release a package like CFFI without extensively testing that it works on a new platform. This is also the approach NumPy has taken: it will not release windows-arm64 wheels without testing. Here is the relevant comment chain on the issue.

I wholeheartedly agree. If you crosscompile, you need some way to test. Where I work this meant completely different system to where build actually happened. Windows ARM64 isn't really yet in great shape. The issue was filed in good faith requesting this eventually since I didn't find existing issue. That is, to signal that consumers needing this exist.

mattip commented 3 weeks ago

The issue was filed in good faith

I am not really a core dev here, but thanks for the clarification. Bootstrapping a new architecture, especially for a platform as popular as windows, is going to take a while.

I think CPython could only release an experimental version because they (a) run a buildbot farm of distributed machines rather than being dependent on centralized CI. This of course has its own security implications, and requires a larger effort. And (b) CPython has the support of the linaro Windows On Arm team who run the CPython buildbot. Maybe someone knows how to talk to them and get some core projects CI runners?

ncoghlan commented 3 weeks ago

Unfortunately, the trust management issues around the core CPython buildbot fleet mean that even offering other projects intermittent access isn't feasible, let alone routine access for artifact creation.

I'm sure MS would like their Copilot+ PCs to be practical platforms for AI development in addition to AI consumption, so I'll see if anyone in the core dev team has GitHub contacts to talk to about community orgs getting access to the ARM64 beta runners.

localden commented 2 weeks ago

@ncoghlan let me know if you don't have any direct contacts and I can likely help connect. We've been looking at deploying some of the auth stack pieces on Windows on ARM (win-arm64) and the cffi dependency blocks us from successfully rolling the code out on the platform.

nitzmahone commented 1 week ago

It's possible this is pending GitHub ARM64 Windows project like for many other Python projects

Yep, exactly.

Windows aarch64 wheels are unlikely to happen anytime soon unless Microsoft does GH-hosted WoA runners like they recently did for MacOS on Apple Silicon. Python wheels with cross-compiled statically-linked extensions are fraught with peril and must be thoroughly tested under their native arches. Access to hardware isn't a problem- I've got plenty of physical/virtual Windows/aarch64 stuff laying around. That said, I can't justify the time to create and maintain a bespoke WoA runner pool. If/when such runners become available to this project, I'm quite happy to add CI and wheel support for WoA.