pip-install-simple with PyQt5 fails on Windows

[jpgill86]

My issue doesn't appear to be a duplicate of #74.

I've attempted to create a minimal example, described below. I'm running Miniconda on Windows.

First get pyoxidizer running with an initialized project:

conda create -n pyoxidizer rust
conda activate pyoxidizer
cargo install pyoxidizer
# add C:\Users\{user}\.cargo\bin to PATH and restart commdand prompt
pyoxidizer init pyapp
pyoxidizer build
pyoxidizer run  # this works fine

Next change pyoxidizer.toml to install PyQt5 using pip (this is just the bare bones config needed to reproduce the bug; I first encountered it when making minimal changes to the default config):

[[build]]
application_name = "pyapp"

[[packaging_rule]]
type = "stdlib-extensions-policy"
policy = "all"

[[packaging_rule]]
type = "stdlib"
include_source = false

[[packaging_rule]]
type = "pip-install-simple"
package = "pyqt5"

[[embedded_python_run]]
mode = "repl"

# END OF COMMON USER-ADJUSTED SETTINGS.
#
# Everything below this is typically managed by PyOxidizer and doesn't need
# to be updated by people.

[[python_distribution]]
build_target = "x86_64-apple-darwin"
url = "https://github.com/indygreg/python-build-standalone/releases/download/20190617/cpython-3.7.3-macos-20190618T0523.tar.zst"
sha256 = "6668202a3225892ce252eff4bb53a58ac058b6a413ab9d37c026a500c2a561ee"
[[python_distribution]]
build_target = "x86_64-pc-windows-msvc"
url = "https://github.com/indygreg/python-build-standalone/releases/download/20190617/cpython-3.7.3-windows-amd64-20190618T0516.tar.zst"
sha256 = "fd43554b5654a914846cf1c251d1ad366f46c7c4d20b7c44572251b533351221"
[[python_distribution]]
build_target = "x86_64-unknown-linux-gnu"
url = "https://github.com/indygreg/python-build-standalone/releases/download/20190617/cpython-3.7.3-linux64-20190618T0324.tar.zst"
sha256 = "d6b80a9723c124d6d193f8816fdb874ba6d56abfb35cbfcc2b27de53176d0620"
[[python_distribution]]
build_target = "x86_64-unknown-linux-musl"
url = "https://github.com/indygreg/python-build-standalone/releases/download/20190617/cpython-3.7.3-linux64-musl-20190618T0400.tar.zst"
sha256 = "2be2d109b82634b36685b89800887501b619ef946dda182e5a8ab5c7029a8136"

[[pyoxidizer]]
version = "0.2.0"
commit = ""

When I try to build again, I get this error:

processing packaging rule: PipInstallSimple(PackagingPipInstallSimple { package: "pyqt5", optimize_level: 0, excludes: [], include_source: true, install_location: Embedded, extra_args: None })
C:\Users\Jeffrey\Desktop\pyapp\build\target\x86_64-pc-windows-msvc\debug\pyoxidizer\python.fd43554b5654\python\install\python.exe: No module named ensurepip
installing modified distutils to C:\Users\Jeffrey\AppData\Local\Temp\pyoxidizer-pip-install.TloZDLxxmb3V\packages
modifying distutils/_msvccompiler.py for oxidation
modifying distutils/command/build_ext.py for oxidation
modifying distutils/unixccompiler.py for oxidation
pip installing to C:\Users\Jeffrey\AppData\Local\Temp\pyoxidizer-pip-install.TloZDLxxmb3V\install
Collecting pyqt5
  ERROR: Could not find a version that satisfies the requirement pyqt5 (from versions: none)
ERROR: No matching distribution found for pyqt5
thread 'main' panicked at 'error running pip', C:\Users\Jeffrey\.cargo\registry\src\github.com-1ecc6299db9ec823\pyoxidizer-0.2.0\src\pyrepackager\packaging_rule.rs:830:9
note: Run with `RUST_BACKTRACE=1` environment variable to display a backtrace.

I guess that PyPI doesn't have a version of PyQt5 compatible with the particular Python distribution listed for Windows in the TOML file:

[[python_distribution]]
build_target = "x86_64-pc-windows-msvc"
url = "https://github.com/indygreg/python-build-standalone/releases/download/20190617/cpython-3.7.3-windows-amd64-20190618T0516.tar.zst"
sha256 = "fd43554b5654a914846cf1c251d1ad366f46c7c4d20b7c44572251b533351221"

I'm not sure why that would be, though, because there is currently a wheel available for Windows AMD 64 Python 3.7:

https://pypi.org/project/PyQt5/#files

PyQt5-5.12.3-5.12.4-cp35.cp36.cp37.cp38-none-win_amd64.whl

[indygreg]

PyOxidizer cannot (yet) use Python wheels with pre-built compiled extension modules because PyOxidizer needs to compile extensions in a certain manner in order to facilitate linking into single file executables. So when PyOxidizer installs Python packages, it must use the source-only distributions, not wheels. This has a number of implications, notably that you must be able to build that Python package locally. This is often non-trivial (unfortunately). But it often just works.

Unfortunately, PyQT is not publishing source-only tarballs in PyPI, only pre-built binary wheels. Why this is, I'm not sure. I consider that a bug against the PyQT project. If nothing else, they should probably be publishing source-only tar.gz files to satisfy licensing requirements that source be made available.

This will be a thorny problem to solve in its entirety, even if PyQT publishes a source distribution. For now, you can try to work around by forcing PyOxidizer to install PyQT from source, possibly by using a setup-py-install rule. But you will likely hit missing dependencies issues on Windows and will need to go down that rabbit hole. I'm sorry I don't have a better solution at this time.

[jpgill86]

That makes sense; thank you for the explanation! If I make any significant progress that yields noteworthy insights, I'll report them here.

[waltonereed]

@indygreg Can you explain how to use a setup-py-install rule to force PyOxidizer to install from source? I'm having the same issue but fortunately the package I want also publishes a source distribution. Thanks!

[matkuki]

Hi guys, Has anyone been successful in pyoxidizing PyQt5 with the procedure mentioned by @indygreg a couple of posts above?

[surfer245]

HI guys,

I have successfully built an executable of a PyQt5-Application with the version 0.10.3 of PyOxidizer for Linux. But when I try to do the same build on Windows 10, everything works fine until the moment when PyQt5.Qtcore is imported. The error message is: "ImportError: DLL load failed while importing QtCore" In my case the PyQt5 module is loaded from the filesystem, i.e. from a subdirectory of the compiled exe. I do not understand why you cannot load an external dll from the compiled python-application. Could someone please shed some light on this topic?

[almarklein]

What if the additional dll's are placed next to the executable?

[surfer245]

What if the additional dll's are placed next to the executable?

I have copied the dlls everywhere, next to the executable or in an extra dictionary. I have even tried to manually add the directory with the dlls via os.add_dll_directory(path) (this is what is happening in the __init__ file of PyQt5) in the REPL (when you omit the python_config.run_*). Nothing works :(

[surfer245]

@mharbison72 was right. The python3.dll was missing, I copied it next to the executable and everything works fine :+1:

[almarklein]

@Surfer298 This is awesome, nice work! Do you have a recipe or something so we can replicate it?

[mharbison72]

@Surfer298 This is awesome, nice work! Do you have a recipe or something so we can replicate it?

There wasn't a release when I tried it, but I installed the commit that fixed it and re-ran the build.

$ cargo install --git https://github.com/indygreg/PyOxidizer.git --rev 2b4e66406f4fafdd7e09bc469878b239dcd55880 pyoxidizer

[surfer245]

@Surfer298 This is awesome, nice work! Do you have a recipe or something so we can replicate it?

well, you have to install pyqt5 as wheel in a subdirectory of the compiled exe, e.g. lib The important commands in the pyoxidzer.bzl are:

policy.resources_location_fallback = "filesystem-relative:lib"
for resource in exe.pip_download(["PyQt5==5.15.1"]):
        resource.add_location = "filesystem-relative:lib"
        exe.add_python_resource(resource)

then you can manually copy the python3.dll next to the compiled exe or use the patch mentioned above.

[liudonghua123]

Is there any updates about this? It's almost 3 years.

Is there any plugins like pyinstaller for packaging pyqt5/6, pyside2/6 app easier?