search

python-cffi / cffi

A Foreign Function Interface package for calling C libraries from Python.
https://cffi.readthedocs.io/en/latest/
Other
114 stars 41 forks source link

Any plans to publish macos `universal2` wheels? #66

Closed carlos-villavicencio-adsk closed 7 months ago

carlos-villavicencio-adsk commented 7 months ago

I'm currently working on a desktop project that bundles CFFI. We're doing the required changes so it can run natively on Apple Silicon processors.

When I got:

ImportError: dlopen(/.../3.10/mac/_cffi_backend.cpython-310-darwin.so, 0x0002): tried: '/.../3.10/mac/_cffi_backend.cpython-310-darwin.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64'))...

Then I realized that CFFI was installed on a macos x86_64 computer but crashes when running on an Apple Silicon (when running without Rosetta).

So, as the title says, are there any plans to publish universal2 wheels? Can you recommend any workaround? I read about some building problems before in #44 .

nitzmahone commented 7 months ago

Sounds like you're conflating Apple Silicon support with universal2 wheels- they're not the same thing.

CFFI has fully supported Apple Silicon for several years, including publishing arch-specific wheels for it. We don't ship fat binary universal2 wheels that support both x86_64 and arm64 concurrently, though- you need to install for one or the other. The standard Python build tools that support universal2 wheels require a single-pass build for both arches, but that's not possible due to libffi's inabilty to generate multi-arch headers. Apple created their own multi-arch headers for their in-box copy of libffi, but dynamic linking to the OS-shipped copy of libffi is very dangerous, since Apple could change that version out at any time. In any case, since single-pass build isn't possible, creating universal2 wheels with fat binaries for CFFI can't use standard build tooling and would have to be done manually.

Since it sounds like you're already shipping a preinstalled environment, if you really need a fat binary, you should be able to make your own without building anything new just by using Apple's lipo tool to stitch together the two copies of (e.g., for Python 3.12) _cffi_backend.cpython-312-darwin.so that we ship in our wheels (one from the Mac x86_64 wheel, one from the Mac arm64 wheel), then replace the Intel copy of that file that's currently installed in your site-packages with the merged fat binary you created. All the rest of the installed file content should be identical between the same version of the x86_64 and arm64 wheels.

carlos-villavicencio-adsk commented 7 months ago

It worked. I appreciate your guidance.