search

pyinstaller / pyinstaller

Freeze (package) Python programs into stand-alone executables
http://www.pyinstaller.org
Other
11.7k stars 1.94k forks source link

Intel Mac app running on M1 Mac: "RuntimeError: found ['./libSDL2_mixer.dylib'], but it's not usable for the library SDL2_mixer" #6410

Open JPLeBreton opened 2 years ago

JPLeBreton commented 2 years ago

I have built an .app bundle of my Python application on my late-2010 Intel-based MacBook Air, using PyInstaller 4.7, Python 3.9.9, SDL2 2.0.16, etc. Here is a link to download the non-notarized build: http://vectorpoem.com/playscii/playscii_mac-9.18.dmg This build runs on other Intel-based Macs. However, a couple of users on M1 Macs are reporting an error relating to the SDL2_mixer libraries - see below for the full log.

The output log running the app from terminal on a user's M1 Mac:

sdl2/dll.py:182: DLLWarning: PyInstallerImportError("Failed to load dynlib/dll './libSDL2_mixer.dylib'. Most likely this dynlib/dll was not found when the application was frozen.")
sdl2/dll.py:182: DLLWarning: PyInstallerImportError("Failed to load dynlib/dll 'libSDL2_mixer.dylib'. Most likely this dynlib/dll was not found when the application was frozen.")
Traceback (most recent call last):
  File "sdl2/sdlmixer.py", line 79, in <module>
  File "sdl2/dll.py", line 184, in __init__
RuntimeError: found ['./libSDL2_mixer.dylib', 'libSDL2_mixer.dylib'], but it's not usable for the library SDL2_mixer

During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "playscii.py", line 31, in <module>
  File "PyInstaller/loader/pyimod03_importers.py", line 476, in exec_module
  File "sdl2/sdlmixer.py", line 82, in <module>
ImportError: found ['./libSDL2_mixer.dylib', 'libSDL2_mixer.dylib'], but it's not usable for the library SDL2_mixer
[25438] Failed to execute script 'playscii' due to unhandled exception: found ['./libSDL2_mixer.dylib', 'libSDL2_mixer.dylib'], but it's not usable for the library SDL2_mixer
[25438] Traceback:
Traceback (most recent call last):
  File "sdl2/sdlmixer.py", line 79, in <module>
  File "sdl2/dll.py", line 184, in __init__
RuntimeError: found ['./libSDL2_mixer.dylib', 'libSDL2_mixer.dylib'], but it's not usable for the library SDL2_mixer

During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "playscii.py", line 31, in <module>
  File "PyInstaller/loader/pyimod03_importers.py", line 476, in exec_module
  File "sdl2/sdlmixer.py", line 82, in <module>
ImportError: found ['./libSDL2_mixer.dylib', 'libSDL2_mixer.dylib'], but it's not usable for the library SDL2_mixer

I see that PyInstaller can in theory build "universal2" fat binary that includes both the Intel and ARM versions of everything needed. But I'm not clear on whether that is the issue here. If it is, then I am curious if it's even possible to build universal apps from an Intel-based Mac - I have no plans to buy an M1, so if the answer is no then I will need to tell my M1 users I can't support them.

JPLeBreton commented 2 years ago

Here's the .spec file used by my project to create the bundle: https://heptapod.host/jp-lebreton/playscii/-/blob/branch/default/playscii_mac.spec

and here's the build script that invokes it: https://heptapod.host/jp-lebreton/playscii/-/blob/branch/default/build_mac.sh

I've never been quite sure if I'm including the SDL2 libraries in the "correct" way there, so it's possible I'm doing multiple things wrong. I would like to know The Right Way but I'm happy to stay within the scope of this issue thread.

bwoodsend commented 2 years ago

Binaries installed via brew are always compiled single architecture so you can't produce a universal2 build.

If it is, then I am curious if it's even possible to build universal apps from an Intel-based Mac

PyInstaller supports universal2 from either architecture in that it doesn't inhibit you from producing them. However, if anything your bundling doesn't do likewise then you're stuffed. How practical universal2 is depends on what your bundling and in your case where your shipping brew libraries then it's not practical at all.

I'm not entirely sure exactly what is causing your crash but it's probably architecture mismatch somewhere. I'm downloading your build now to see if I can find it but honestly I think that your sanest options are:

rokm commented 2 years ago

FWIW, I don't think this issue is related to M1; the linked build fails to run on my test x86_64 systems, too, with the same error. And it looks like the problem is that the bundled libSDL2_mixer-2.0.0.dylib depends on libmpg123.0.dylib, which is not bundled.

JPLeBreton commented 2 years ago

FWIW, I don't think this issue is related to M1; the linked build fails to run on my test x86_64 systems, too, with the same error. And it looks like the problem is that the bundled libSDL2_mixer-2.0.0.dylib depends on libmpg123.0.dylib, which is not bundled.

As per my suspicion above, I generated a new build using nothing from brew, just using a locally installed Python 3.9.9 universal2 from python.org, and then installing "pysdl2-dll" via pip to take care of the dependencies. This let me get rid of the manual includes to specific brew-installed library paths in that .spec linked above, and just let PyInstaller do its thing - it's advanced a lot in the 5-6 years since I made that initial setup, and it produced an .app without any noticeable errors.

Here's the new build, I'm curious as to if it runs on your system (or someone's M1 Mac obviously): http://vectorpoem.com/playscii/playscii_mac-9.18c.dmg

rokm commented 2 years ago

I don't think SDL libraries were collected into this new build...

JPLeBreton commented 2 years ago

I haven't forgotten about this thread, I'm just taking the time to make sure that I can even get a working 64 bit Intel Mac bundle built with the needed sdl2 dependencies (without using Homebrew) before I try troubleshooting anything M1 related.

rokm commented 2 years ago

Hmm, I suspect it might be easier to just collect the libmpg123.0.dylib from homebrew in addition to all libraries that were already collected than to switch to pysdl2-dll. From target arch perspective it makes no difference, because pysdl2-dll also provides only x86_64 shared libs, so you'll end up with x86_64-only build either way.

pysdl2-dll on macOS provides shared libs in .framework bundles, which are not properly handled by PyInstaller. Perhaps the easiest way is to just collect them all as data files, for example using a custom hook:

hook-sdl2dll.py

```python # hook-sdl2dll.py from PyInstaller.compat import is_win, is_darwin from PyInstaller.utils.hooks import collect_dynamic_libs, collect_data_files # pysdl2-dll is available only on Windows and macOS. if is_win: # On Windows, we can collect the DLLs normally, preserving the sdl2dll\dll sub-directory layout. binaries = collect_dynamic_libs('sdl2dll') elif is_darwin: # Collect the shared library .framework bundles as datas. datas = collect_data_files('sdl2dll') ```

Because if shared libraries from those .framework bundles end up collected as binaries, the library path rewriting is going to kick in, and then things will get very messy because it assumes that the libraries are located directly in top-level directory, which in this case they are not... The downside of collecting everything as a data file is that shared libraries do not unergo automatic re-signing, but you are doing a onedir build and likely have a post-processing signing step, so that's not really an issue.

JPLeBreton commented 2 years ago

Thanks for the info, I had zero idea about .framework bundles not being supported by PyInstaller, I'll try the workaround you describe here. If Brew is never going to be able to do multi-arch builds, then I'll need to find another way - I might try filing an issue with the pysdl2-dll project to see how feasible it'd be for them to offer multi-arch libs.

Are there any other examples of multi-platform, SDL2-based applications out there that use PyInstaller? I might have naively assumed that there were plenty besides mine years ago.