PyInstaller executable does not find Qt WebEngine Process

0xWOLAND commented 5 months ago

I have a project that uses PyQt6, specifically its QtWebSockets server and I have built it with Pyinstaller with windowed mode activated. If I extract the executable from a zipped target file produced by Tufup, it works. However, if I try to let Tufup update an older version of the package and run the newly produced executable, PyQt throws:

The following paths were searched for Qt WebEngine Process:
  /Helpers/QtWebEngineProcess.app/Contents/MacOS/QtWebEngineProcess
  /.../my_app.app/Contents/MacOS/QtWebEngineProcess
but could not find it.
You may override the default search path by using QTWEBENGINEPROCESS_PATH environment variable.

To Reproduce Steps to reproduce the behavior:

Build tufup-example using QtWebSockets

Expected behavior I expect the executable built by the auto-updater to work the same way as the target zip package that contains the PyQt app with the latest version.

System info (please complete the following information):

OS: MacOS
OS Version Sonoma 14.5
Python version 3.9

Any help is much appreciated! Please let me know if I can provide any more details

0xWOLAND commented 5 months ago

I'm not sure if this is relevant

dennisvang commented 5 months ago

@0xWOLAND Thanks for posting.

At first glance, this looks like a configuration issue, perhaps related to the PyInstaller spec, or perhaps a macOS-specific issue. I guess your link to pyinstaller/pyinstaller#2276 could indeed be relevant.

However, your statement

If I extract the executable from a zipped target file produced by Tufup, it works. However, if I try to let Tufup update an older version of the package and run the newly produced executable, PyQt throws [...]

Suggests there may be something else going on.

In essence, tufup is just a fancy file-mover. It does not do anything with the contents of the app bundle, other than copy them into the location you specify as app_install_dir. So, tufup itself does not know (nor care) that your app uses QtWebSockets.

The basic steps taken by the update client on macOS are as follows:

check for new update
download new update (if available), i.e. either a full archive or a patch
in case of patch: patch current archive to reconstruct new archive
extract new archive into temporary dir https://github.com/dennisvang/tufup/blob/4bb16ad228b4b6f39e42527cc4367e501d728142/src/tufup/client.py#L316-L318
move extracted files from temporary dir to app_install_dir https://github.com/dennisvang/tufup/blob/4bb16ad228b4b6f39e42527cc4367e501d728142/src/tufup/utils/platform_specific.py#L259
restart the app https://github.com/dennisvang/tufup/blob/4bb16ad228b4b6f39e42527cc4367e501d728142/src/tufup/utils/platform_specific.py#L265-L266

Perhaps something is going wrong during one of the last steps on macOS...

Some questions to get more insight:

Which version of tufup are you using?
Could you share the complete PyInstaller call, so we can see exactly which options are used, etc.?
Are tufup patches enabled? https://github.com/dennisvang/tufup/blob/4bb16ad228b4b6f39e42527cc4367e501d728142/src/tufup/client.py#L143
Did you try manually starting the executable after the update was applied?
Did you inspect the content of the app_install_dir and compare it with the content of the manually-extracted archive, i.e. the one that did work?
Could you be a bit more specific in the "steps to reproduce"? A minimal example of the app code including instantiation of the QWebSocketServer would be helpful.

dennisvang commented 5 months ago

Another thing you could try:

Manually remove the contents of the app_install_dir, then extract the content of your "working" tufup archive into that dir, and try to run the executable from there.

0xWOLAND commented 5 months ago

Here is my spec file: main.txt. I built my package with patch = False because it took a very long time to build, maybe this is the source of the issue? I'm currently rebuilding with patch enabled. What does patch do?

Manually remove the contents of the app_install_dir, then extract the content of your "working" tufup archive into that dir, and try to run the executable from there.

If I do this, the executable works.

0xWOLAND commented 5 months ago

Update: doesn't work with patch enabled

0xWOLAND commented 5 months ago

If I run diff -r on the newly formed executable and the newest version in targets/repository, the packages are identical.

0xWOLAND commented 5 months ago

It seems like the file myapp.app/Contents/Frameworks/PySide6/Qt/lib/QtWebEngineCore.framework/Helpers/QtWebEngineProcess.app/Contents/MacOS/QtWebEngineProcess exists, not sure why the executable isn't checking here.

dennisvang commented 5 months ago

Here is my spec file: main.txt. I built my package with patch = False because it took a very long time to build, maybe this is the source of the issue? I'm currently rebuilding with patch enabled. What does patch do?

@0xWOLAND: tufup supports two types of updates:

full, i.e. patch=False: for every update, the complete new archive (.tar.gz) is downloaded
patch, i.e. patch=True: the new archive is reconstructed by applying a binary patch to the current archive (which is already available on the client side)

Patch update files are typically much smaller, and therefore much quicker to download. The only drawback is that creation of the binary patch, on the repo side, takes a long time and a lot of memory, as explained in #105. However, you only need to do that once for every release. Patch application, on the client side, is (relatively) fast.

After patch application, the resulting archive must be byte-by-byte identical to the new archive. This is verified by checking the length and hash. So, in the end, as long as the patch procedure succeeds, it should not matter whether you do a patch update or a full update, because you end up with the exact same file bundle.

If you do see a difference in operation between full mode and patch mode, perhaps some error is being silenced inadvertently. Could you post the command line output of a failed update attempt on the client-side, with logging level set to DEBUG? (make sure to remove any sensitive info)

dennisvang commented 5 months ago

It seems like the file myapp.app/Contents/Frameworks/PySide6/Qt/lib/QtWebEngineCore.framework/Helpers/QtWebEngineProcess.app/Contents/MacOS/QtWebEngineProcess exists, not sure why the executable isn't checking here.

@0xWOLAND Just to be sure: Did you try setting QTWEBENGINEPROCESS_PATH as suggested in the original error message?

PyInstaller<6.0 used to have a specific runtime hook doing just that, but this has been removed in pyinstaller 6.0.

See pyinstaller/pyinstaller#7619 and the pyinstaller 6.0 changelog.

Also see understanding-pyinstaller-hooks and changing-runtime-behavior for more info.

In any case, pyinstaller has a long history of issues related to QtWebEngine as discussed e.g. here.

dennisvang commented 5 months ago

Another question: what is the exact pyinstaller command you are using? For example, do you use --windowed and --onefile together?

dennisvang commented 5 months ago

...

Manually remove the contents of the app_install_dir, then extract the content of your "working" tufup archive into that dir, and try to run the executable from there.

If I do this, the executable works.

Seeing that your pyinstaller bundle does appear to work when you do it manually, but then does not work after tufup copies it into place, suggests that something strange may be going on with the paths. For example, an absolute path issue, or something with relative paths that do not take into account the app being in a subfolder.

Tufup does not modify the files or directories inside your app bundle, but it does determine where the bundle itself is located.

Did you verify that the directory structure in the app_install_dir was exactly the same after the manual procedure above and after the automatic update?

0xWOLAND commented 5 months ago

...

Manually remove the contents of the app_install_dir, then extract the content of your "working" tufup archive into that dir, and try to run the executable from there.

If I do this, the executable works.

Seeing that your pyinstaller bundle does appear to work when you do it manually, but then does not work after tufup copies it into place, suggests that something strange may be going on with the paths. For example, an absolute path issue, or something with relative paths that do not take into account the app being in a subfolder.

Tufup does not modify the files or directories inside your app bundle, but it does determine where the bundle itself is located.

Did you verify that the directory structure in the app_install_dir was exactly the same after the manual procedure above and after the automatic update?

I ran diff -r on the two directories (newly built one and the newest version in target/, and the directory structures are identical

dennisvang commented 5 months ago

I ran diff -r on the two directories (newly built one and the newest version in target/, and the directory structures are identical

Not sure this is what I'm looking for. Let me clarify:

Try the automatic update:

install your old app version
let your app update using tufup
run the updated app (manually) with logging level set to DEBUG, and verify that the error is raised
post the command line output here (make sure to remove any sensitive info) (!)
take note of the resulting directory structure in app_install_dir

Then do a "manual install":

manually remove all content of app_install_dir
manually extract the content of the new archive from your tufup targets dir into the app_install_dir
make sure the resulting directory structure is an exact match to the previously noted one, including any nesting
run the app and ensure the error is not raised

In any case, I cannot do much else without seeing a complete, minimal reproducible example. Note the emphasis on complete and minimal. ;-)

It also helps to start reproducing the issue from a completely clean slate.

0xWOLAND commented 5 months ago

Hey, I'm currently working on building a minimal reproducible example. I think I tried manually setting the QTWEBENGINEPROCES_PATH as

os.environ["QTWEBENGINEPROCESS_PATH"] = os.path.normpath(
    os.path.join(
        sys._MEIPASS,
        "PySide6",
        "Qt",
        "lib",
        "QtWebEngineCore.framework",
        "Helpers",
        "QtWebEngineProcess.app",
        "Contents",
        "MacOS",
        "QtWebEngineProcess",
    )
)
os.environ["QTWEBENGINE_RESOURCES_PATH"] = os.path.normpath(
    os.path.join(
        sys._MEIPASS,
        "..",
        "Resources",
        "PySide6",
        "Qt",
        "lib",
        "QtWebEngineCore.framework",
        "Resources",
    )
)
os.environ["QTWEBENGINE_LOCALES_PATH"] = os.path.normpath(
    os.path.join(
        sys._MEIPASS,
        "..",
        "Resources",
        "PySide6",
        "Qt",
        "lib",
        "QtWebEngineCore.framework",
        "Resources",
        "qtwebengine_locales",
    )
)

which seems to get rid of the issue. However, a new issue arises:

[87489:86787:0609/221319.040978:ERROR:child_process_launcher_helper_mac.cc(166)] Failed to compile sandbox policy: empty subpath pattern
[87489:86787:0609/221319.155533:ERROR:child_process_launcher_helper_mac.cc(166)] Failed to compile sandbox policy: empty subpath pattern

Do you have any thoughts on how to resolve this?

dennisvang commented 5 months ago

@0xWOLAND This again looks like something related to PyInstaller, QtWebEngine, and Chromium on macOS.

Perhaps you can find some clues in one of the following issues/prs:

pyinstaller/pyinstaller#6903
qutebrowser/qutebrowser#7278
ocaml/opam#3806

There's also an unanswered question on SO that may be related, although this is not very helpful.

Just to verify: In your original post you mentioned using PyQt6, but now I see you are using PySide6? Their APIs may be nearly identical, but it's an important detail.

0xWOLAND commented 5 months ago

Thanks for the quick reply! Here's the minimal example

0xWOLAND commented 5 months ago

I forgot to mention that the same behavior as earlier occurs, except this error takes place instead.

dennisvang commented 5 months ago

Thanks for the quick reply! Here's the minimal example

That's a good start, but minimal implies remove everything that is not required to reproduce the issue.

For example:

do you need the whole React app, or could you reproduce the issue with just a plain HTML file?
does the error occur if you leave out entire backend package? (e.g. visit example.com instead)
and what if you leave out the DeskTopApp part?
and what if you only import the QWebEngineView and instantiate it without calling load()?
...

and so on and so forth, until you have the absolute minimum necessary to reproduce the issue.

0xWOLAND commented 5 months ago

Looks like simply disabling sandbox (following the docs) works. All I had to do is os.environ["QTWEBENGINE_DISABLE_SANDBOX"] = "1". However, yet another bug has spawned 😭. It seems like the relative paths are messed when tufup moves the package to app_install_dir.

dyld[40500]: Library not loaded: @rpath/QtWebEngineCore
  Referenced from: 
 /Users/USER/Applications/myapp/myapp Runtime.app/Contents/Frameworks/PySide6/Qt/lib/QtWebEngineCore.framework/Helpers/QtWebEngineProcess.app/Contents/MacOS/QtWebEngineProcess
  Reason: tried: 
'/Users/USER/Applications/myapp/myapp Runtime.app/Contents/Frameworks/PySide6/Qt/lib/QtWebEngineCore.framework/Helpers/QtWebEngineProcess.app/Contents/MacOS/../../../../../../../../../../QtWebEngineCore' (no such file), 
'/Users/USER/Applications/myapp/myapp Runtime.app/Contents/Frameworks/PySide6/Qt/lib/QtWebEngineCore.framework/Helpers/QtWebEngineProcess.app/Contents/MacOS/../../../../../../../../../../QtWebEngineCore' (no such file), 
'/usr/local/lib/QtWebEngineCore' (no such file),
'/usr/lib/QtWebEngineCore' (no such file, not in dyld cache)
dyld[40501]: Library not loaded: @rpath/QtWebEngineCore
  Referenced from: 
 /Users/USER/Applications/myapp/myapp Runtime.app/Contents/Frameworks/PySide6/Qt/lib/QtWebEngineCore.framework/Helpers/QtWebEngineProcess.app/Contents/MacOS/QtWebEngineProcess
  Reason: tried: 
'/Users/USER/Applications/myapp/myapp Runtime.app/Contents/Frameworks/PySide6/Qt/lib/QtWebEngineCore.framework/Helpers/QtWebEngineProcess.app/Contents/MacOS/../../../../../../../../../../QtWebEngineCore' (no such file), 
'/Users/USER/Applications/myapp/myapp Runtime.app/Contents/Frameworks/PySide6/Qt/lib/QtWebEngineCore.framework/Helpers/QtWebEngineProcess.app/Contents/MacOS/../../../../../../../../../../QtWebEngineCore' (no such file),
'/usr/local/lib/QtWebEngineCore' (no such file), 
'/usr/lib/QtWebEngineCore' (no such file, not in dyld cache)

dennisvang commented 5 months ago

As mentioned above, tufup only does shutil.unpack_archive(...) followed by shutil.copytree(...).

It's not immediately clear to me how that would cause this relative path issue.

Can you locate QtWebEngineCore manually in the tree under app_install_dir, before and after the update?

0xWOLAND commented 5 months ago

The files that exist are:

./myapp Runtime.app/Contents/Resources/PySide6/Qt/lib/QtWebEngineCore.framework/Versions/A/QtWebEngineCore
./myapp Runtime.app/Contents/Resources/PySide6/Qt/lib/QtWebEngineCore.framework/Versions/Current/QtWebEngineCore
./myapp Runtime.app/Contents/Resources/PySide6/Qt/lib/QtWebEngineCore.framework/QtWebEngineCore
./myapp Runtime.app/Contents/Resources/QtWebEngineCore
./myapp Runtime.app/Contents/Frameworks/PySide6/Qt/lib/QtWebEngineCore.framework/Versions/A/QtWebEngineCore
./myapp Runtime.app/Contents/Frameworks/PySide6/Qt/lib/QtWebEngineCore.framework/Versions/Current/QtWebEngineCore
./myapp Runtime.app/Contents/Frameworks/PySide6/Qt/lib/QtWebEngineCore.framework/QtWebEngineCore
./myapp Runtime.app/Contents/Frameworks/QtWebEngineCore

But the path

'/Users/USER/Applications/myapp/myapp Runtime.app/Contents/Frameworks/PySide6/Qt/lib/QtWebEngineCore.framework/Helpers/QtWebEngineProcess.app/Contents/MacOS/../../../../../../../../../../QtWebEngineCore' (no such file), '/Users/USER/Applications/myapp/myapp Runtime.app/Contents/Frameworks/PySide6/Qt/lib/QtWebEngineCore.framework/Helpers/QtWebEngineProcess.app/Contents/MacOS/../../../../../../../../../../QtWebEngineCore' (no such file), '/usr/local/lib/QtWebEngineCore'

doesn't exist if I check manually. The long path above is equivalent to /Users/USER/Applications/myapp/myapp Runtime.app/

Relevant: https://github.com/pyinstaller/pyinstaller/issues/2276#issuecomment-280921074

dennisvang commented 5 months ago

Sounds more and more like an exotic issue that is out-of-scope for tufup.

Anyway, I don't have access to a mac (other than github runners) so cannot reproduce.

Perhaps these can help:

dennisvang commented 5 months ago

So it's looking for a path relative to the location of QtWebEngineProcess (which you specified using QTWEBENGINEPROCESS_PATH)

./myapp Runtime.app/Contents/Frameworks/PySide6/Qt/lib/QtWebEngineCore.framework/Helpers/QtWebEngineProcess.app/Contents/MacOS/QtWebEngineProcess

with all the ../../../ and so on that results in

./myapp Runtime.app/QtWebEngineCore

but the actual file is in

./myapp Runtime.app/Contents/Frameworks/QtWebEngineCore

The difference is Contents/Frameworks/, which is not explicitly part of your QTWEBENGINEPROCESS_PATH.

Not sure if that has anything to do with it, but it stands out.

What's the output of sys._MEIPASS?

0xWOLAND commented 5 months ago

What's the output of sys._MEIPASS?

/Users/USER/Applications/myapp/myapp Runtime.app/Contents/Frameworks

0xWOLAND commented 5 months ago

Looks like it was https://github.com/dennisvang/tufup/blob/4bb16ad228b4b6f39e42527cc4367e501d728142/src/tufup/utils/platform_specific.py#L259 throws the problem because symlink aren't handled. It works if symlinks=True.

0xWOLAND commented 5 months ago

@dennisvang I am happy to close this issue

dennisvang commented 5 months ago

Looks like it was

https://github.com/dennisvang/tufup/blob/4bb16ad228b4b6f39e42527cc4367e501d728142/src/tufup/utils/platform_specific.py#L259 throws the problem because symlink aren't handled. It works if symlinks=True.

@0xWOLAND Good find! :)

dennisvang commented 5 months ago

@dennisvang I am happy to close this issue

@0xWOLAND It would probably be convenient to expose the symlinks argument in the tufup api.

I'll keep the issue open until that's resolved.

dennisvang commented 5 months ago

@0xWOLAND I've prepared #148 with the option to enable symlinks as follows:

client.download_and_apply_updates(..., symlinks=True)

Perhaps you could give that a try?

dennisvang commented 5 months ago

@0xWOLAND The symlinks option is now available in tufup 0.9.0 (via pypi)

0xWOLAND commented 5 months ago

@0xWOLAND I've prepared #148 with the option to enable symlinks as follows:
client.download_and_apply_updates(..., symlinks=True)
Perhaps you could give that a try?

Yep it works!

dennisvang commented 5 months ago

@0xWOLAND Thanks! That's good to know. :)

dennisvang / tufup

PyInstaller executable does not find Qt WebEngine Process #145