Creating application with matplotlib, numpy, and scipy -- .app fails to run

[the-user-created]

I am using Pycharm Professional on Catalina 10.15.5

I am importing the following packages:

from tkinter import Scrollbar, Frame, Canvas, Label, StringVar, font, Checkbutton, BooleanVar, Entry, Tk
from numpy import dot, linalg, vstack, ones, array
from scipy.stats import linregress
from math import log10, floor
from matplotlib.backends.backend_tkagg import FigureCanvasTkAgg
from matplotlib.figure import Figure
from matplotlib import use

If I only include tkinter and math the .app runs fine but without the features I need.

My setup.py file:

from setuptools import setup

APP = ['main.py']
DATA_FILES = []
OPTIONS = {}

setup(
    app=APP,
    data_files=DATA_FILES,
    options={'py2app': OPTIONS},
    setup_requires=['py2app'],
)

I run python3 setup.py py2app --packages=matplotlib --packages=numpy --packages=scipy

Setup runs fine.

Open main.app and I get the following popup:

I also have two questions:

1. Is there a better way to run the python3 setup.py py2app #some packages command?
2. What about adding the packages to setup.py?

Thanks :)

[ronaldoussoren]

You can add the packages to the setup.py file, but... that should be needed if the packages are installed

setup(
     ....
    options={
       "py2app": {
          "packages": [ "matplotlib", ...]
      }
   }
)

The more interesting issue is why the app won't launch. What error do you get if you start the application from the command line (run "dist/main.app/Contents/MacOS/main")? Starting from the command-line should give a better error message that the dialog.

[the-user-created]

Thanks for your reply :) I have just opened the application using "sudo open /dist/main.app/Contents/MacOS/main" and get the following error:

Traceback (most recent call last):
  File "/Users/USERNAME/PycharmProjects/PROJECT/dist/main.app/Contents/Resources/__boot__.py", line 89, in _recipes_pil_prescript
    import Image
ModuleNotFoundError: No module named 'Image'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/USERNAME/PycharmProjects/PROJECT/dist/main.app/Contents/Resources/__boot__.py", line 136, in <module>
    _recipes_pil_prescript(['TiffImagePlugin', 'PpmImagePlugin', 'PngImagePlugin', 'IcnsImagePlugin', 'SpiderImagePlugin', 'MicImagePlugin', 'FliImagePlugin', 'SunImagePlugin', 'XVThumbImagePlugin', 'XbmImagePlugin', 'FtexImagePlugin', 'WebPImagePlugin', 'BufrStubImagePlugin', 'PdfImagePlugin', 'Hdf5StubImagePlugin', 'PixarImagePlugin', 'Jpeg2KImagePlugin', 'McIdasImagePlugin', 'TgaImagePlugin', 'SgiImagePlugin', 'ImtImagePlugin', 'MpoImagePlugin', 'BlpImagePlugin', 'MpegImagePlugin', 'PalmImagePlugin', 'XpmImagePlugin', 'DdsImagePlugin', 'GifImagePlugin', 'FitsStubImagePlugin', 'FpxImagePlugin', 'IcoImagePlugin', 'PcxImagePlugin', 'CurImagePlugin', 'GbrImagePlugin', 'PcdImagePlugin', 'JpegImagePlugin', 'WmfImagePlugin', 'ImImagePlugin', 'GribStubImagePlugin', 'IptcImagePlugin', 'PsdImagePlugin', 'MspImagePlugin', 'DcxImagePlugin', 'EpsImagePlugin', 'BmpImagePlugin'])
  File "/Users/USERNAME/PycharmProjects/PROJECT/dist/main.app/Contents/Resources/__boot__.py", line 93, in _recipes_pil_prescript
    from PIL import Image
  File "<frozen zipimport>", line 259, in load_module
  File "PIL/Image.pyc", line 94, in <module>
  File "<frozen zipimport>", line 259, in load_module
  File "PIL/_imaging.pyc", line 14, in <module>
  File "PIL/_imaging.pyc", line 10, in __load
  File "imp.pyc", line 342, in load_dynamic
ImportError: dlopen(/Users/USERNAME/PycharmProjects/PROJECT/dist/main.app/Contents/Resources/lib/python3.8/lib-dynload/PIL/_imaging.so, 2): Library not loaded: @loader_path/.dylibs/libjpeg.9.dylib
  Referenced from: /Users/USERNAME/PycharmProjects/PROJECT/dist/main.app/Contents/Resources/lib/python3.8/lib-dynload/PIL/_imaging.so
  Reason: image not found
2020-09-12 07:52:30.641 main[1187:16099] main Error

[Process completed]

So it seems that the "Image" is not compiling into the .app correctly.

What do you think?

EDIT: I have just added "PIL" to the packages in setup.py and the .app now works. I can't believe the answer to my problem was sitting right in front of me :/

If you do not mind I also have another question. My code is only 451 lines long and you have seen my imports. Yet I have an application size of 285,8 MB... I am wanting the application to be less than 50 MB for ease of distribution. Is there any way to reduce this size?

[ronaldoussoren]

That's weird, PIL/Imaging should just work. What version of py2app do you use?

The error message indicates that the PIL/Imaging package isn't copied correctly, particularly the C shared libaries used by the _imaging extension.

[the-user-created]

I am using py2app 0.21

Here all the installed packages:

[ronaldoussoren]

I need to investigate further, but...:

Don't use "sudo ..." to start the application, this runs the application with elevated privileges and is not necessary. In general it is better to avoid using sudo unless there is a clear reason for doing so. It is possible to do Python development without using the sudo command.

My setup:

• I install Python using the Python.org installer, which uses admin privileges to do the install (the same can, but doesn't have to be, true for other installation methods, like homebrew or anaconda)
• I use virtual environments (either using "python3 -m ven" or the "virtualenv" package) to have isolated interpreters per project
• I don't use system installs of Python (including the copy of Python3 installed with Xcode)

[the-user-created]

Hi again,

I have completed my program and it has been working splendidly on my own Mac. However one of my friends who I asked to test the .app has just sent me the error message I was getting 6 days ago...

I am using github to share this application and when I download the application it runs fine... however now on two other macs it has been failing to start up because of this error.

Any idea why the imaging library is still working fine on my mac but not on theirs?

[Kerbidiah]

I am getting an identical error message with a similar setup. I am also using tkinter and NumPy, but I am using Plotly instead of matPlotLib and I am also using pandas. However, I use Homebrew to install python (3.9). I am using PyCharm CE, and I am using macOS big sur. The app is also really large for me as well, 425 MB, but about 74 MB of that is a couple CSV files that need to go with the app (as long as it works this is not a problem). I have tried launching the app from terminal and normally to no avail.

Also when compiling, I get the following error: File "/usr/local/lib/python3.9/site-packages/PyQt5/uic/port_v2/ascii_upper.py", line 27, in <module> _ascii_trans_table = string.maketrans(string.ascii_lowercase, AttributeError: module 'string' has no attribute 'maketrans' Maybe this has something to do with it?

[ronaldoussoren]

The @loader_path problem is fixed in 2fde755a903bb9f20c3095b41f1ef4088f435690.

Could you check if the problem is fixed when you install py2app from this git repository?

My current plan is to release a new version of py2app before the end of the year.

[TomChen-catseye]

The @loader_path problem is fixed in 2fde755.

Could you check if the problem is fixed when you install py2app from this git repository?

My current plan is to release a new version of py2app before the end of the year.

I encountered a problem similar to that of the main post. After some testing and tossing, I have a general understanding of the source of the problem. The problem is divided into two main aspects: 1) In some integrated development tool suites, independently packaged python environments and resource packages are used; When compiling, the program py2app does not have the right to access and extract resource packages in these integrated environments, which leads to the problem of importing resource packages; In the following failed test example, py2app failed when trying to obtain the python3 resource package integrated by Thonny.APP eg.

 from tkinter import \*
 File "/Applications/Thonny.app/Contents/Frameworks/Python.framework/Versions/3.7/lib/python3.7/tkinter/\_\_init\_\_.py", line 36, in <module>
 import \_tkinter # If this fails your Python may not be configured for Tk
ImportError: dlopen(/Applications/Thonny.app/Contents/Frameworks/Python.framework/Versions/3.7/lib/python3.7/lib-dynload/\_tkinter.cpython-37m-darwin.so, 2): Library not loaded: @rpath/Python.framework/Versions/3.7/lib/libtcl8.6.dylib
 Referenced from: /Applications/Thonny.app/Contents/Frameworks/Python.framework/Versions/3.7/lib/python3.7/lib-dynload/\_tkinter.cpython-37m-darwin.so
 Reason: image not found

2) It may be that the directory permissions established in the integrated environment are not system permissions, leading to some obstacles when compiling and reading the setup.py file, including generating the app files. This obstacle can be bypassed when trying to directly raise the permissions to root, but it will cause confusion in the permissions of the entire directory. Because the project directory is created by the user and the dist directory is created by root, there are some contradictions in later transactions.

In addition, adding the description of "packages": ["matplotlib", ...]in the setup() section in the setup.py editor according to your description will cause overall failure. The icon and version information will be invalid. It is not clear why. here is my setup.py:

from setuptools import setup

APP = ['Just.py']
DATA_FILES = []
OPTIONS = {
    "argv_emulation": True,
    "excludes": ['Pillow', 'Image'],
    'iconfile':'Radiobug.icns',
    'plist': {
        'CFBundleName': 'Just',
        'CFBundleDisplayName': 'Just',
        'CFBundleGetInfoString': "Just",
        'CFBundleIdentifier': "Apple Development: XXXXXXXXX",
        'CFBundleVersion': "0.0.2",
        'CFBundleShortVersionString': "0.0.2",
        'NSHumanReadableCopyright': u"Copyright © 2021, Tom Chen, All Rights Reserved"
        }
    }

setup(
    app=APP,
    data_files=DATA_FILES,
    options={
        'py2app': {
            "packages": ["tkinter"]
            }
    },
    setup_requires=['py2app'],
)

The entire main program is just a sample Tkinter program, with only one window and one exit button, and it is only used by me to verify compilation and release.

At last I can succeed by putting all the compiling operations in the python3 environment that comes with Xcode12 without the description of "packages": ["tkinter"].

I am just a rookie who has been using Python for six months. If I have any mistakes or omissions, please point them out directly, or tell me how to correct them. Please send me any suggestions!

[ronaldoussoren]

The Thonny issue you mention is due to the use of @rpath in MachO binaries from Thonny. That might be fixable in a similar way to how I fixed the @loader_path issue, but that's something I'll have to explore.

The icon problem is due to unclear communication on my end. It should start working again when you make two changes:

1. add 'packages': ['tkinter'], to the definition of OPTIONS
2. use options=OPTIONS, in the call to setup instead of what's there now

That's because the "packages=..." bit is an additional configuration option for py2app, the way its now you've replaced the configuration options entirely.

That said, packages=['tkinter'], shoudn't be necessary, in general py2app should be able to find dependencies automatically. The packages option is only necessary when the automatic detection doesn't work (for example when a package) is only imported from a C extension, or when a package doesn't work in a zipfile (which you'll notice at runtime). There is a similar option for modules/extensions as well. Feel free to report when you run into a package on PyPI that requires using these options, I can add detection logic to py2app to automatically set these options (as I've already done for a number of packages).

BTW. Thanks for the reference to Thonny, that looks like an interesting project, and gives me an easy way to reproduce the @rpath issue.

P.S. Good luck with learning Python! There's a pythonmac-sig maillist for Python users on macOS, although that list has been pretty silent the last couple of years.

[TomChen-catseye]

The Thonny issue you mention is due to the use of @rpath in MachO binaries from Thonny. That might be fixable in a similar way to how I fixed the @loader_path issue, but that's something I'll have to explore. BTW. Thanks for the reference to Thonny, that looks like an interesting project, and gives me an easy way to reproduce the @rpath issue.

P.S. Good luck with learning Python! There's a pythonmac-sig maillist for Python users on macOS, although that list has been pretty silent the last couple of years.

Thank you for your detailed explanation of this matter, which has allowed me to learn more.