_webp.cpython-312-darwin.so is not a fat binary error building a universal2 app

[danrossi]

I'm trying to bundle PIL via PyInstaller into a universal2 app on macOS ventura. I have Python 3.12.1 universal2 installed. I've installed Pillow via pipenv

pipenv install Pillow

_to_target_arch
    raise IncompatibleBinaryArchError(f"{display_name} is not a fat binary!")
PyInstaller.utils.osx.IncompatibleBinaryArchError: /path/virtualenvs/multicast-agent-tlrJmy6M/lib/python3.12/site-packages/PIL/_webp.cpython-312-darwin.so is not a fat binary!

Do I need to install Pillow a particular way to have a universal binary for webp ? It seems it's an arm64 libwebp installed ? My app doesn't even use or need webp so if there is a way to filter it out it might help.

Hopefully this is the correct place to report or PyInstaller.

[radarhere]

Hi. If you go to https://pillow.readthedocs.io/en/stable/installation.html#basic-installation, and open the 'macOS' tab, you will find instructions on how to create a universal2 Pillow wheel. I suggest creating that wheel, installing it, and then trying again.

[danrossi]

I'm sorry for missing some documentation. I will try that. However I am installing with pipenv virtualenv not to pip. I'll have to figure out how to do that.

python3 -m pip download --only-binary=:all: --platform macosx_10_10_x86_64 Pillow
python3 -m pip download --only-binary=:all: --platform macosx_11_0_arm64 Pillow
python3 -m pip install delocate

[radarhere]

The python3 -m pip download commands are just a simple way of downloading the wheels to disk. If that method isn't your preference, you can just go to https://pypi.org/project/Pillow/#files and download the relevant wheels by hand.

[radarhere]

If it helps, here's a universal2 wheel I just created from the instructions - Pillow-10.1.0-cp312-cp312-macosx_11_0_universal2.whl.zip

[danrossi]

I see I can't keep the version up to date with a pipenv check and pipenv update command. For instance PIL has had security notices requiring updates I usually do before repackaging with PyInstaller. So won't be able to know if it needs an update.

I built a script to automate this but don't know how to automate getting the wheel bundle into pipenv yet

import subprocess
import tempfile
import glob
import os
from delocate.fuse import fuse_wheels
from pkginfo import Wheel

pwd = os.path.dirname(__file__)

print(pwd)

with tempfile.TemporaryDirectory() as tmp_dir:

    amd64_pil = "macosx_10_10_x86_64"
    arm64_pil = "macosx_11_0_arm64" 
    subprocess.check_call(['python3', '-m', 'pip', 'download', '--only-binary=:all:','--platform', amd64_pil, 'Pillow', '-d', tmp_dir])
    subprocess.check_call(['python3', '-m', 'pip', 'download', '--only-binary=:all:','--platform', arm64_pil, 'Pillow', '-d', tmp_dir])
    universal_wheels = glob.glob("{0}/*".format(tmp_dir))
    wheel = Wheel(universal_wheels[0])
    universal2_wheel = "{0}/Pillow-{1}-cp39-cp39-macosx_11_0_universal2.whl".format(pwd,wheel.version)
    fuse_wheels(*universal_wheels, universal2_wheel)
    print("Successfully created universal2 wheel ", universal2_wheel)

[danrossi]

I tried to install by running

pipenv install build/mac/Pillow-10.1.0-cp39-cp39-macosx_11_0_universal2.whl

and got this error

[pipenv.exceptions.ResolutionFailure]: Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
  You can use $ pipenv run pip install <requirement_name> to bypass this mechanism, then run $ pipenv graph to inspect the versions actually installed in the virtualenv.
  Hint: try $ pipenv lock --pre if it is a pre-release dependency.
ERROR: Pillow-10.1.0-cp39-cp39-macosx_11_0_universal2.whl is not a supported wheel on this platform.

[danrossi]

According to this

from distutils import util
util.get_platform()

macOS Ventura outputs

macosx-10.9-universal2

So the wheel needs to be named that?

[radarhere]

You have said that you are using Python 3.12.

When you run this command,

subprocess.check_call(['python3', '-m', 'pip', 'download', '--only-binary=:all:','--platform', amd64_pil, 'Pillow', '-d', tmp_dir])

I suspect that python3 is Python 3.9, and so you're downloading the wrong version of the wheel?

Installing a Python 3.9 wheel into Python 3.12 will not work.

[danrossi]

No it's python 3.12.1

python3 --version

I ran the commands in the docs manually and got the same. The default python universal2 installer detects Ventura platform as 10.9.

pip install Pillow-10.1.0-cp39-cp39-macosx_11_0_universal2.whl
ERROR: Pillow-10.1.0-cp39-cp39-macosx_11_0_universal2.whl is not a supported wheel on this platform.

[radarhere]

Could you try renaming the file to Pillow-10.1.0-cp312-cp312-macosx_11_0_universal2.whl?

[danrossi]

That is correct I apologise. I'll update my script to parse the version that was the problem.

[danrossi]

The updated automation script that works is this

import subprocess
import tempfile
import glob
import os
from delocate.fuse import fuse_wheels 
from wheel_filename import parse_wheel_filename

cwd = os.path.dirname(__file__)

with tempfile.TemporaryDirectory() as tmp_dir:

    amd64_pil = "macosx_10_10_x86_64"
    arm64_pil = "macosx_11_0_arm64"
    subprocess.check_call(['python', '-m', 'pip', 'download', '--only-binary=:all:','--platform', amd64_pil, 'Pillow', '-d', tmp_dir])
    subprocess.check_call(['python', '-m', 'pip', 'download', '--only-binary=:all:','--platform', arm64_pil, 'Pillow', '-d', tmp_dir])
    universal_wheels = glob.glob("{0}/*".format(tmp_dir))
    wheel = parse_wheel_filename(universal_wheels[0])
    universal2_wheel = os.path.join(cwd, "Pillow-{0}-{1}-{1}-macosx_11_0_universal2.whl".format(wheel.version, wheel.python_tags[0]))
    fuse_wheels(*universal_wheels, universal2_wheel)
    print("Successfully created universal2 wheel ", universal2_wheel)

When I install the wheel using pipenv it overrides the pillow entry for Windows. I have to figure out how to allow both.

[radarhere]

If I understand correctly, your remaining problem relates to PyInstaller? Is there anything left that we can help you with using Pillow knowledge?

[danrossi]

It's ok. I guess provide that helper script might help others. I had to fuse 2 other packages to bundle an app. I have it working. I've improved it into a class.

I had to do some hacks for pipenv to have both packages installed.

Pillow = { version = "*", markers = "platform_system == 'Windows'" }
pyobjc = { version = "*", markers = "platform_system == 'Darwin'" }
pillow = {file = "build/mac/Pillow-10.1.0-cp312-cp312-macosx_11_0_universal2.whl", markers = "platform_system == 'Darwin'" }

class Universal2Bundler:

    def build(self, dest_dir, package):
        with tempfile.TemporaryDirectory() as tmp_dir:

            amd64_binary = "macosx_10_10_x86_64"
            arm64_binary = "macosx_11_0_arm64"
            subprocess.check_call(['python', '-m', 'pip', 'download', '--only-binary=:all:','--no-deps','--platform', amd64_binary, package, '-d', tmp_dir])
            subprocess.check_call(['python', '-m', 'pip', 'download', '--only-binary=:all:','--no-deps','--platform', arm64_binary, package, '-d', tmp_dir])
            universal_wheels = glob.glob("{0}/*".format(tmp_dir))

            wheel = parse_wheel_filename(universal_wheels[0])
            universal2_wheel = os.path.join(dest_dir, "{0}-{1}-{2}-{2}-macosx_11_0_universal2.whl".format(package, wheel.version, wheel.python_tags[0]))
            fuse_wheels(*universal_wheels, universal2_wheel)
            print("Successfully created universal2 wheel ", universal2_wheel)

bundler = Universal2Bundler()
bundler.build(cwd, "pillow")
bundler.build(cwd, "cffi")
bundler.build(cwd, "websockets")

[duhiqc37]

Does anyone have a universal version of PyQt5?