search

pypa / cibuildwheel

🎡 Build Python wheels for all the platforms with minimal configuration.
https://cibuildwheel.pypa.io
Other
1.79k stars 229 forks source link

Cross compilation yields x86-64 wheel #1722

Open m8dotpie opened 5 months ago

m8dotpie commented 5 months ago

Description

We are trying to compile wheel for casadi wrapped with cmeel, but with aarch64 support. To avoid using arm machine as well as emulation (which I was unable to correctly enable in current cmeel setup), we decided to move for dockcross/manylinux2014-aarch64. The problem is cibuildwheel correctly detects target architecture and, as can be seen in the logs, tells starts building cp310-manylinux_aarch64 wheel. However, resulting wheel is still for x86_64

Build log

https://pastebin.com/rhfTC9qv

CI config

https://github.com/m8dotpie/cmeel-casadi/blob/aarch64/.github/workflows/build_aarch64.yml

henryiii commented 5 months ago

There has been a lot of talk about supporting this (see our issues); the biggest issue is not actually related to the build backend, but the compilers: no one uses the native compilers for manylinux, but always the RHEL-patched modern compilers. These are especially important on special architectures. NumPy actually did a lot of work to push for a newer manylinux image, then found they couldn't even use that since they required a newer compiler on ARM due to bugfixes.

Given the attempts to do this usually hitting this wall then failing, we haven't pushed it through yet.

FYI, scikit-build-core hasn't had any issues with emulation AFAIK. I've heard py-build-cmake is good at cross compiling, but I don't think anyone has figured out a way around the issue I've pointed out above, other than using another language (Rust). I heard the Zig compiler might be a working solution, but haven't tried it yet.

FYI, https://scikit-build-core.readthedocs.io/en/latest/#other-projects-for-building covers some of the known backends.

nim65s commented 1 month ago

@m8dotpie : distributing binary cmeel wheels compatible with each others is hard. It also require each dependency to be distributed independently. Don't hesitate to get in touch with @lvjonok and I for this.

m8dotpie commented 1 month ago

@lvjonok and I for this.

Actually me and Lev know each other and been fighting this for a while, but did not find feasible solution. Use case is compilation of the cmeel wheels of Lev's casadi and casadi-kin-dyn for macos-universal (arm64) or at least aarch64 (since it is posssible to dual-boot macs), but we did not manage to accomplish this

Actually I did not manage to build the packages using cmeel locally, not speaking about the CI.