Data files aren't copied correctly with pathlib, or is it preferred to use some other path type?

[goodzack]

I have a program (using a tkinter GUI) that has some data stored in .txt and .json files that are stored in a "resources" folder like this:

├── main.py
├── resources
│   ├── json
│   │   └── data.json
│   └── textfiles
│       ├── textfile 1.txt
│       └── textfile 2.txt
└── setup.py

The way the files are accessed is with pathlib relative to __file__:

textfile_path = Path(__file__).parent / 'resources/textfiles/textfile 1.txt'
other_path = Path('resources/textfiles/textfile 2.txt')
json_path = Path(__file__).parent / 'resources/json/data.json'

Here's the way the data files are included in the setup.py:

from setuptools import setup

APP = ['main.py']
DATA_FILES = []
OPTIONS = {}

setup(
    app=APP,
    data_files=[('textfiles', ['./resources/textfiles/']),
                ('json', './resources/json/')],
    options={'py2app': OPTIONS},
    setup_requires=['py2app'],
)

The .app I get won't run in finder (gives the "launch error" pop-up). It does the same thing when I run it from the terminal (open main.app and doesn't put anything in the console. When I run it from open dist/main.app/Contents/MacOS/main I get this in the console:

FileNotFoundError: [Errno 2] No such file or directory: '/Users/goodzack/code/practice/py2app-troubleshoot/dist/main.app/Contents/Resources/resources/textfiles/textfile.txt'
2022-04-14 15:12:55.417 main[73293:4727721] Launch error
2022-04-14 15:12:55.417 main[73293:4727721] Launch error
See the py2app website for debugging launch issues

I see that there might be some issues with pathlib (#436, #418), especially when using __file__. Could that be the problem here?

[goodzack]

I didn't want to put this in one big post, but for reference, here is the code for a working program:

import json
from collections import ChainMap
from pathlib import Path
import tkinter as tk
from tkinter import ttk

textfile_path = Path(__file__).parent / 'resources/textfiles/textfile.txt'
other_path = Path('resources/textfiles/textfile.txt')
json_path = Path(__file__).parent / 'resources/json/data.json'

class MainWindow(ttk.Frame):
    def __init__(self, parent, *args, **kwargs):
        ttk.Frame.__init__(self, parent)
        self.parent = parent

        text_file_one_frame = ttk.Frame(self)
        text_file_one_text = ''.join([i for i in open(textfile_path, 'r')])
        text_file_one_label = ttk.Label(text_file_one_frame, text=text_file_one_text)
        text_file_one_label.pack()
        text_file_one_frame.pack(padx=5, pady=10)

        text_file_two_frame = ttk.Frame(self)
        text_file_two_text = ''.join([i for i in open(other_path, 'r')])
        text_file_two_label = ttk.Label(text_file_two_frame, text=text_file_two_text)
        text_file_two_label.pack()
        text_file_two_frame.pack(padx=5, pady=10)

        json_file_frame = ttk.Frame(self)
        json_file_list = json.load(open(json_path, 'r')).get('big data')
        json_file_dict = dict(ChainMap(*json_file_list))
        json_file_text = '\n'.join(["key: " + i + ", value: " + json_file_dict.get(i) for i in json_file_dict])

        json_file_label = ttk.Label(json_file_frame, text=json_file_text)
        json_file_label.pack()
        json_file_frame.pack(padx=5, pady=10)

root = tk.Tk()
root.title('title')
mainwindow = MainWindow(root)
mainwindow.pack()
root.mainloop()

this is what is in ./resources/json/data.json:

{"big data":
    [{"data 1" :"one"},
    {"data 2" : "two"},
    {"data 3" : "three"}]}

this is what is in ./resources/textfiles/textfile 1.txt:

test line one
test line two
test line three

this is what is in ./resources/textfiles/textfile 2.txt:

second file test line one
second file test line two
second file test line three

and it should open a window with something like this:

[goodzack]

One last thing: I've messed around with some of the parameters in the real program I'm working on and I can't remember what it was exactly, but I managed to get it to run in terminal using --no-chdir, so I think it was actually using my relative paths, but then it wouldn't run from the finder, probably because of #263.

Anyways, the main thing I want to know is whether this is a bug or whether there is a single best way of doing this, because a lot of the description in the documentation and all has been pretty sparse

[ronaldoussoren]

I don't think this is related to pathlib, especially when it works with "--no-chdir".

Have to checked if the resource files are actually copied in the app bundle at the location you expect them to be?

[ronaldoussoren]

I had to do some minor adjustments to main.py to get a working script (both Path objects loading a text file refer to same file), and likewise to setup.py:

from setuptools import setup

APP = ['main.py']
DATA_FILES = []
OPTIONS = {}

setup(
    app=APP,
    data_files = ['resources'],
    options={'py2app': OPTIONS},
    setup_requires=['py2app'],
)

By just including the entire 'resources' folder in the app it works. I got a build error with your initial version due an error while trying to copy. That seems to be a bug in py2app itself that I haven't debugged yet.

BTW. I used the py2app version from the repo, which has some code cleanup and modernisation compared to the last release. Those may have broken this.

[goodzack]

Sorry about the error there -- thought I posted the fixed version, but I'm glad you caught it.

I tried pointing this like yours (at just ['resources']) and it works in the minimal working example but not in the real app. In that one, the 'resources' folder is ending up in main.app/Contents/Resources/ For what

Unfortunately, I realize now there's a slightly different error in the actual app, which I wish I posted earlier above:

NotADirectoryError: [Errno 20] Not a directory: '/Users/goodzack/code/hulqtransliterator/dist/maingui.app/Contents/Resources/lib/python310.zip/resources/graphemes/graphemes.json'
2022-04-15 11:37:07.465 maingui[92247:5638878] Launch error
2022-04-15 11:37:07.465 maingui[92247:5638878] Launch error

In the minimum working example I posted above I thought it'd be the same error, but it's not for some reason, and I'm not sure why the real app is zipping where the minimal one is not.

In the real app, the data files are being put in at maingui.app/Contents/Resources/lib/python3.10/ (so there's maingui.app/Contents/Resources/lib/python3.10/[resources folder with data files found here]

but if you unzip the folder, it's not in maingui.app/Contents/Resources/lib/python310/, which is where it's looking for it.