Application based off Gooey GUI won't launch after being packaged

[f-tonini]

I successfully packaged a python application that relies on the Gooey for GUI portion. Below is the code I used inside the installer.cfg file.

[Application]
name=SAVI Tool
version=1.0a

# How to launch the app - this calls the 'main' function from the 'myapp' package:
entry_point=app.main:main
icon=img/program_icon.ico

[Python]
version=3.6.6
bitness=64
format=bundled

[Include]
# Packages from PyPI that your application requires, one per line
# These must have wheels on PyPI:
packages=app
local_wheels = wheels/*.whl
pypi_wheels = Gooey==1.0.1
    scikit-learn==0.19.0
    pandas==0.23.4
    numpy==1.15.1

# To bundle packages which don't publish wheels, or to include directly wheel files
# from a directory, see the docs on the config file.

# Other files and folders that should be installed
files = LICENSE
    sample_data/
    frag_models/

The resulting packaged folder is shown in the attached image.

If I double-click on the .exe file, the application installs inside the ./Program Files folder with name equal to the application name (in my case SAVI tool).

How can I now start the application and the GUI? The SAVI_Tool.launch.pyw file does not seem to launch/open anything. Am I doing something wrong?

[takluyver]

There should be a 'SAVI Tool' shortcut in the start menu. If that doesn't do anything, have a look for exceptions in a log file:

https://pynsist.readthedocs.io/en/latest/installers.html#uncaught-exceptions

[f-tonini]

Thanks for the tip. I tried launching the application as per the instructions in the link you pasted above.

C:\Program Files\Application>Python\python.exe "Application.launch.pyw"

To show tracebacks in the command prompt. It seems like the error is shows currently is:

Traceback (most recent call last):
  File "C:\Program Files\SAVI Tool\SAVI_Tool.launch.pyw", line 31, in <module>
    from app.main import main
  File "C:\Program Files\SAVI Tool\pkgs\app\main.py", line 1, in <module>
    import os, shutil, errno, utils
ModuleNotFoundError: No module named 'utils'

The module "utils" is simply a script I created that's called by the main.py script. When I bundled the application to create the .exe file, I put both main.py and my utils.py inside a folder called "app". See my code in the posting at the beginning.

[Include]
packages=app

What am I doing wrong? How can I bundle my application including both main.py and utils.py that is called from the main.py script using "import utils"?

[takluyver]

You'll need to make app a package by adding a __init__.py file - this can be an empty file if you don't need to put anything there, but it needs to exist.

Then to import another file in the same package, you use either of these:

# Absolute import
from app import utils

# Relative import
from . import utils

[f-tonini]

Thank you Thomas.

I now get the following error:

Traceback (most recent call last):
  File "C:\Program Files\SAVI Tool\SAVI_Tool.launch.pyw", line 31, in <module>
    from app.main import main
  File "C:\Program Files\SAVI Tool\pkgs\app\main.py", line 4, in <module>
    import rasterio as rio
  File "C:\Program Files\SAVI Tool\pkgs\rasterio\__init__.py", line 31, in <module>
    from rasterio._base import gdal_version
ImportError: DLL load failed: The specified module could not be found.

which is strange given that I provide both rasterio 1.0.9 and GDAL 2.3.2 as .whl files from Christoph Gohlke's wheels repository..both for python 3.6 and 64bit. Is this type of error related to what is packaged inside the wheels? I could try contacting Mr. Gohlke if you think that is the best way.

[takluyver]

I can't help you much with the specifics, but I know that that error (DLL load failed: The specified module could not be found.) often means that a package you're loading requires some DLL that's missing. A program called dependency walker can sometimes help you figure it out, but it's a bit fiddly. I think GDAL is a fairly big and complex compiled library that's just being wrapped into Python, so there's a good chance this isn't something straightfoward, unfortunately.

Christoph Gohlke might be able to help you figure it out. He builds a lot of packages for Windows; I don't know if he's got any special interest in GDAL.

A useful test, if you haven't already, would be to pip install the same GDAL package into a clean Python installation on a clean Windows computer, and see if the same error occurs. If it does, then it's certainly something you can take up with Christoph Gohlke. If not, it's something to do with how Pynsist bundles it into the application.

[takluyver]

Ah, looking into the GDAL wheel, it puts part of the package in the .data directory in a way that Pynsist currently ignores. That's very likely the problem. Either Pynsist needs to extract that, or the wheel needs to be built with the files arranged differently.

@cgohlke is there a reason the files in wheel end up at GDAL-2.3.2.data/data/Lib/site-packages/ rather than in the wheel root or in GDAL-2.3.2.data/platlib/?

Here's the code Pynsist currently uses for handling the .data directory:

https://github.com/takluyver/pynsist/blob/4bf55cbd6aad405f0d6e3d7127779a32b8d851b7/nsist/wheels.py#L233-L239

[takluyver]

Thinking about it, I think that a wheel built that way will fail with a --user install, because you'll end up with some files in %APPDATA%\Python\Python3.6\site-packages (the correct location for a --user install on Windows) and some in %APPDATA%\Python\Lib\site-packages.

So before making changes in Pynsist to work around it, I'd like to understand how the wheel is getting built that way and whether it's practical for it to be arranged differently. If this is a common pattern, maybe we have to make Pynsist work with it. But if it's something peculiar that @cgohlke has done for one or a few packages, I think it might be best to fix it in the process of building the wheels.

[cgohlke]

is there a reason the files in wheel end up at GDAL-2.3.2.data/data/Lib/site-packages/

That's where setuptools/wheel puts data_files=[('Lib/site-packages/osgeo', glob('../../gdal*.dll'))].

[takluyver]

Thanks! I'd usually use package_data for files that are meant to go inside a package. Is there a reason to use data_files instead and hardcode Lib/site-packages? Is this a pattern you use for a lot of packages?

Also, am I right that these wheels will break if you use pip install --user on them, or have I misunderstood something?

[cgohlke]

AFAIK package_data files have to be located inside/under the package directory, but in GDAL and other primarily C/C++ libraries, the DLLs and data files are located outside the Python package directory. Copying files in setup.py into the package directory is tedious. Using data_files with distutils/setuptools, I don't know how to make sure the data files are installed into packagedir other than specifying Lib/site-packages/packagedir. I have never used pip install --user. Let me know if there is a better way to pack data files located outside the Python package directory.

[takluyver]

Fair enough. I don't know of a better option without writing instructions to copy the files into the package.

From reading code, I'm about 80% sure that wheels built this way will break on a --user install because the site-packages dir there is under e.g. a Python36 folder, so the files the package needs will be installed in the wrong place. I don't have a Windows installation handy to test this on.

It's up to you whether you care about this, of course. If you do, then maybe the easiest solution is to postprocess the wheels to rearrange the paths inside the zip file.

[f-tonini]

Thank you Thomas and @cgohlke for following up on this. Please let me know what solution you come up with that would allow all users of Pynsist to bundle applications into executables that make use of GDAL to read/write spatial data.

[takluyver]

@f-tonini I think I'll have to work around it. Can you try building an installer with PR #172 and see if it solves the problem? Let me know if you need pointers on installing from git.

[f-tonini]

Yes, please send pointers. I am not sure how to build the installer from PR #172.

[takluyver]

# 1. Clone the repository
git clone https://github.com/takluyver/pynsist.git
cd pynsist

# 2. Checkout the branch
git checkout wheels-data-lib

# 3. Install pynsist
pip install flit
flit install

Then cd to your application directory and use Pynsist as normal to build your installer.

If you're not familiar with git, an alternative to steps is to download a zip file and unpack it, then start a command prompt in the unpacked folder. Here's the URL for a zip download of that branch: https://github.com/takluyver/pynsist/archive/wheels-data-lib.zip

[f-tonini]

Yes, I am familiar with git, I just was not sure how to install the latest changes you made since they were not merged into the master branch. Do I have to run both: pip install flit and flit install instead of the normal pip install pynsist?

[takluyver]

You'll need pip install flit unless you have Flit installed already. Then flit install inside the Pynsist folder replaces the normal pip install pynsist.

[f-tonini]

I followed your instructions and re-built my application. When debugging the installed application with:

C:\Program Files\Application>Python\python.exe "Application.launch.pyw"

I was getting an error:

Traceback (most recent call last):
  File "C:\Users\Francesco Tonini\Dropbox\GITHUB-Projects\SAVI-tool\build\nsis\SAVI_Tool.launch.pyw", line 31, in <module>
    from app.main import main
  File "C:\Users\Francesco Tonini\Dropbox\GITHUB-Projects\SAVI-tool\build\nsis\pkgs\app\main.py", line 4, in <module>
    import rasterio as rio
  File "C:\Users\Francesco Tonini\Dropbox\GITHUB-Projects\SAVI-tool\build\nsis\pkgs\rasterio\__init__.py", line 31, in <module>
    from rasterio._base import gdal_version
  File "rasterio\_base.pyx", line 27, in init rasterio._base
  File "C:\Users\Francesco Tonini\Dropbox\GITHUB-Projects\SAVI-tool\build\nsis\pkgs\rasterio\env.py", line 3, in <module>
    import attr
ModuleNotFoundError: No module named 'attr'

I tried adding attr to the installer.cfg pypi_wheels list but get this:

Unpacking Python...
Copying packages into build directory...
Using cached wheel: C:\Users\Francesco Tonini\AppData\Local\pynsist\pypi\Gooey\1.0.1\Gooey-1.0.1-py2.py3-none-any.whl
Using cached wheel: C:\Users\Francesco Tonini\AppData\Local\pynsist\pypi\scikit-learn\0.19.0\scikit_learn-0.19.0-cp36-cp36m-win_amd64.whl
Using cached wheel: C:\Users\Francesco Tonini\AppData\Local\pynsist\pypi\pandas\0.23.4\pandas-0.23.4-cp36-cp36m-win_amd64.whl
Using cached wheel: C:\Users\Francesco Tonini\AppData\Local\pynsist\pypi\numpy\1.15.1\numpy-1.15.1-cp36-none-win_amd64.whl
Using cached wheel: C:\Users\Francesco Tonini\AppData\Local\pynsist\pypi\pytz\2018.7\pytz-2018.7-py2.py3-none-any.whl
Using cached wheel: C:\Users\Francesco Tonini\AppData\Local\pynsist\pypi\python-dateutil\2.7.5\python_dateutil-2.7.5-py2.py3-none-any.whl
Using cached wheel: C:\Users\Francesco Tonini\AppData\Local\pynsist\pypi\six\1.11.0\six-1.11.0-py2.py3-none-any.whl
Using cached wheel: C:\Users\Francesco Tonini\AppData\Local\pynsist\pypi\click\7.0\Click-7.0-py2.py3-none-any.whl
Traceback (most recent call last):
  File "d:\anaconda3\envs\accusim\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "d:\anaconda3\envs\accusim\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "D:\Anaconda3\envs\AccuSim\Scripts\pynsist.exe\__main__.py", line 9, in <module>
  File "d:\anaconda3\envs\accusim\lib\site-packages\nsist\__init__.py", line 518, in main
    ec = InstallerBuilder(**args).run(makensis=(not options.no_makensis))
  File "d:\anaconda3\envs\accusim\lib\site-packages\nsist\__init__.py", line 471, in run
    self.prepare_packages()
  File "d:\anaconda3\envs\accusim\lib\site-packages\nsist\__init__.py", line 344, in prepare_packages
    wg.get_all()
  File "d:\anaconda3\envs\accusim\lib\site-packages\nsist\wheels.py", line 295, in get_all
    self.get_requirements()
  File "d:\anaconda3\envs\accusim\lib\site-packages\nsist\wheels.py", line 301, in get_requirements
    whl_file = wl.fetch()
  File "d:\anaconda3\envs\accusim\lib\site-packages\nsist\wheels.py", line 182, in fetch
    return self.get_from_pypi()
  File "d:\anaconda3\envs\accusim\lib\site-packages\nsist\wheels.py", line 147, in get_from_pypi
    raise NoWheelError('No compatible wheels found for {0.name} {0.version}'.format(self))
nsist.wheels.NoWheelError: No compatible wheels found for attr 0.3.1

This is strange since attr==0.3.1 is actually listed on PyPI. Any workaround? I am guessing I am hitting a bunch of dependencies issues since some of the wheels do not have everything bundled and installed like Conda does when I was testing my application locally with the appropriate libraries.

[takluyver]

In that case, I think it's simply that the package you want is attrs with an s. It's a bit confusing because its install name and its import name differ.

(If you're curious, the attr project only has a wheel for Python 2 - that's why Pynsist isn't finding it when you build a Python 3 installer)

[f-tonini]

How do you find out which module correctly matches the name of the wheel that needs to be imported? I am having similar issues with 'backports' now where I get the same error above.

[takluyver]

You can either examine each package individually - e.g. rasterio specifies its requirements here: https://github.com/mapbox/rasterio/blob/6647a9d94a51886b1a14f4c760535458ec2566b1/setup.py#L333-L335

Or you can make an empty directory and run pip wheel rasterio in it, and pip should download wheels of all the Python packages rasterio requires. Those are from PyPI; for things like GDAL it might be useful to use Christoph Gohlke's wheel instead, because he bundles the C library into it.

Anyway, it sounds like you're making progress, so I'll go ahead with the branch. Thanks for testing.

[f-tonini]

I will write here once I am able to finally reproduce the GUI and test it after install to make sure GDAL and the other packages have been bundled correctly.

[f-tonini]

Ok, getting closer. I now get this error. It is related to Gooey not finding the program icon which I originally had inside the .img folder in the root of my application. I also tried with ../img as well as moving the /img folder inside the app package folder where main.py is now.

Traceback (most recent call last):
  File "C:\Users\Francesco Tonini\Dropbox\GITHUB-Projects\SAVI-tool\build\nsis\SAVI_Tool.launch.pyw", line 32, in <module>
    main()
  File "C:\Users\Francesco Tonini\Dropbox\GITHUB-Projects\SAVI-tool\build\nsis\pkgs\gooey\python_bindings\gooey_decorator.py", line 83, in inner2
    return payload(*args, **kwargs)
  File "C:\Users\Francesco Tonini\Dropbox\GITHUB-Projects\SAVI-tool\build\nsis\pkgs\app\main.py", line 85, in main
    args = parser.parse_args()
  File "C:\Users\Francesco Tonini\Dropbox\GITHUB-Projects\SAVI-tool\build\nsis\pkgs\gooey\python_bindings\gooey_parser.py", line 113, in parse_args
    return self.parser.parse_args(args, namespace)
  File "C:\Users\Francesco Tonini\Dropbox\GITHUB-Projects\SAVI-tool\build\nsis\pkgs\gooey\python_bindings\gooey_decorator.py", line 78, in run_gooey
    application.run(build_spec)
  File "C:\Users\Francesco Tonini\Dropbox\GITHUB-Projects\SAVI-tool\build\nsis\pkgs\gooey\gui\application.py", line 15, in run
    app = build_app(build_spec)
  File "C:\Users\Francesco Tonini\Dropbox\GITHUB-Projects\SAVI-tool\build\nsis\pkgs\gooey\gui\application.py", line 23, in build_app
    imagesPaths = image_repository.loadImages(build_spec['image_dir'])
  File "C:\Users\Francesco Tonini\Dropbox\GITHUB-Projects\SAVI-tool\build\nsis\pkgs\gooey\gui\image_repository.py", line 26, in loadImages
    return {'images': merge(defaultImages, collectOverrides(targetDir, filenames))}
  File "C:\Users\Francesco Tonini\Dropbox\GITHUB-Projects\SAVI-tool\build\nsis\pkgs\gooey\gui\image_repository.py", line 42, in collectOverrides
    targetDir))
OSError: Unable to find the user supplied directory img

my folder structure is:

- img/
- installer.cfg
- app ---------------- main.py
                     __init__.py
                     utils.py
                     img/

My installer.cfg looks like:

[Application]
name=SAVI Tool
version=1.0a

# How to launch the app - this calls the 'main' function from the 'app' package:
entry_point=app.main:main
icon=img/program_icon.ico

[Python]
version=3.6.6
bitness=64
format=bundled

[Include]
# Packages from PyPI that your application requires, one per line
# These must have wheels on PyPI:
packages=app
local_wheels = wheels/*.whl
pypi_wheels = Gooey==1.0.1
    scikit-learn==0.19.0
    pandas==0.23.4
    numpy==1.15.1
    pytz==2018.7
    python-dateutil==2.7.5
    six==1.11.0
    click==7.0
    attrs==18.2.0
    affine==2.2.1
    wxPython==4.0.3
    Pillow==5.3.0
    scipy==1.1.0
    backports.csv==1.0.6

# To bundle packages which don't publish wheels, or to include directly wheel files
# from a directory, see the docs on the config file.

# Other files and folders that should be installed
files = LICENSE
    sample_data/
    frag_models/

I believe something must go wrong after pynsist bundles the app and the gooey pkg, losing the image icon connection? Inside my main.py file, I have:

@Gooey(dump_build_config=True, 
       program_name="Simulation Accuracy and Validation Informatics (SAVI)", 
       **image_dir="img"**,
       default_size=(715, 550)
       )

[takluyver]

I haven't looked in detail, but I'd guess that expects img in the working directory, which it won't be when you launch your application from the start menu. Have a look at Using data files in the FAQ.

[f-tonini]

Thomas, everything seems to work now and the application bundles and installs correctly. Before we close this issue though, there is an odd behavior in the bundled tool. When I test-run the tool with

C:\Program Files\Application>Python\python.exe "Application.launch.pyw"

The tool runs and write the output of any print() statement onto the console GUI. However, if I run the tool from the Windows Start menu, after having it installed with the .exe file, the tool runs but nothing gets written to the output console. Is this intended behavior? Can you think of any reason why the print() statement would not work when run full-fledge separately?

[takluyver]

You might want to use console=true in the config file (docs). If you don't, it will start the application without a console, and output is written to the log file you checked to discover the errors.

[f-tonini]

Perfect! I am going to close the issue for now, as it has been solved at the moment. Thank you for putting so much effort and getting back to me quickly on solving it. May I ask if you know of any similar great tool like yours for bundling Mac OS and Linux applications like mine? Thanks!

[takluyver]

The alternatives section in the FAQ lists a few cross-platform tools. Naturally I think they're not as good as Pynsist ;-), but Pyinstaller is quite popular and actively developed.

I also know of py2app for Macs, but I don't have much experience using it.

I'd like to work at some point on application distribution for Linux, but I haven't got round to it yet.