Bug report: pyface.qt initialization failing to find installed PySide2 package.

capn-freako commented 1 year ago

Environment

OS: Windows 11 Python version: 3.9 Toolkit: Qt Qt API: PySide2

Description

My Traits/UI application is failing to launch with the following back-trace:

Traceback (most recent call last):
  File "C:\Users\capnf\anaconda3\envs\pybert-dev\lib\site-packages\pyface\base_toolkit.py", line 219, in import_toolkit
    toolkit_object = plugin.load()
  File "C:\Users\capnf\anaconda3\envs\pybert-dev\lib\importlib\metadata.py", line 86, in load
    module = import_module(match.group('module'))
  File "C:\Users\capnf\anaconda3\envs\pybert-dev\lib\importlib\__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "C:\Users\capnf\anaconda3\envs\pybert-dev\lib\site-packages\pyface\ui\qt4\init.py", line 17, in <module>
    from pyface.qt import QtGui
  File "C:\Users\capnf\anaconda3\envs\pybert-dev\lib\site-packages\pyface\qt\__init__.py", line 48, in <module>
    raise ImportError("Cannot import any of " + ", ".join(modules))
ImportError: Cannot import any of PySide2, PySide6, PyQt5, PyQt6, PyQt4

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\capnf\anaconda3\envs\pybert-dev\lib\runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\capnf\anaconda3\envs\pybert-dev\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Users\capnf\Documents\GitHub\PyBERT\src\pybert\__main__.py", line 6, in <module>
    from pybert.gui.view import traits_view
  File "C:\Users\capnf\Documents\GitHub\PyBERT\src\pybert\gui\view.py", line 14, in <module>
    from enable.component_editor import ComponentEditor
  File "C:\Users\capnf\anaconda3\envs\pybert-dev\lib\site-packages\enable\component_editor.py", line 13, in <module>
    from enable.colors import ColorTrait
  File "C:\Users\capnf\anaconda3\envs\pybert-dev\lib\site-packages\enable\colors.py", line 13, in <module>
    from pyface.toolkit import toolkit
  File "C:\Users\capnf\anaconda3\envs\pybert-dev\lib\site-packages\pyface\toolkit.py", line 23, in <module>
    toolkit = toolkit_object = find_toolkit("pyface.toolkits")
  File "C:\Users\capnf\anaconda3\envs\pybert-dev\lib\site-packages\pyface\base_toolkit.py", line 265, in find_toolkit
    return import_toolkit(ETSConfig.toolkit, entry_point)
  File "C:\Users\capnf\anaconda3\envs\pybert-dev\lib\site-packages\pyface\base_toolkit.py", line 231, in import_toolkit
    raise RuntimeError(msg) from toolkit_exception
RuntimeError: No pyface.toolkits plugin could be loaded for qt4

But, the PySide2 package is installed:

$ conda list pyside2
# packages in environment at C:\Users\capnf\anaconda3\envs\pybert-dev:
#
# Name                    Version                   Build  Channel
pyside2                   5.15.8           py39h56cbfbc_2    conda-forge

Steps to Reproduce

Install PyBERT, as per: installation instructions.
Set _ETSTOOLKIT environment variable to: "qt4".
Launch PyBERT. (either: pybert or python -m pybert)

corranwebster commented 1 year ago

Not sure what's going on here - I assume that import PySide2.QtCore works in this environment? We have seen occasional issues where the official wheels of PySide don't work on older Python versions (although it looks like you're using conda). I probably can't diagnose without knowing what happens when you try to import PySide2, since that's almost certainly what is causing the failure.

Other things to check would be:

whether there is any other Qt wrapper in the environment - that can be made to work, but you need to be careful
whether anything else is importing Qt before Pyface - it shouldn't matter, but it would be a data point

A possible work-around is to set the QT_API environment variable to pyside2 - at the very least it will force a different code path which may give a more informative error.

I will add an issue to add some logging information to the Qt selection code so at least the error might be more helpful in the future.

corranwebster commented 1 year ago

Opened #1205 for better debug logging.

capn-freako commented 1 year ago

Hi @corranwebster ,

Thank you! for your always prompt responses to my issues. I really appreciate that. :)

I assume that import PySide2.QtCore works in this environment?

No!:

(pybert-dev)
capnf@DESKTOP-G84ND7C MINGW64 ~
$ python -c "import PySide2.QtCore"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: DLL load failed while importing QtCore: The specified module could not be found.

What does this failure imply about my environment?

The import of just PySide2 does work:

(pybert-dev)
capnf@DESKTOP-G84ND7C MINGW64 ~
$ python -c "import PySide2"
(pybert-dev)
capnf@DESKTOP-G84ND7C MINGW64 ~
$

Is that significant?

In case it's helpful:

(pybert-dev)
capnf@DESKTOP-G84ND7C MINGW64 ~
$ conda list qt
# packages in environment at C:\Users\capnf\anaconda3\envs\pybert-dev:
#
# Name                    Version                   Build  Channel
pyqt                      5.15.7           py39hb08f45d_0    conda-forge
pyqt5-sip                 12.11.0          py39h415ef7b_0    conda-forge
pyqtgraph                 0.13.1           py39haa95532_0
qt-main                   5.15.8               h720456b_6    conda-forge
qtpy                      2.2.0            py39haa95532_0

$ python -c "import traits as dut; print(dut.__version__)"
6.2.0

$ python -c "import traitsui as dut; print(dut.__version__)"
7.2.1

$ python --version
Python 3.9.16

capn-freako commented 1 year ago

A possible work-around is to set the QT_API environment variable to pyside2 - at the very least it will force a different code path which may give a more informative error.

This doesn't seem to add anything new to the output:

(pybert-dev)
capnf@DESKTOP-G84ND7C MINGW64 ~
$ QT_API=pyside2 python -c "import PySide2.QtCore"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: DLL load failed while importing QtCore: The specified module could not be found.

capn-freako commented 1 year ago

Hi @corranwebster ,

Seeing the last comment/suggestion on this post, I tried:

pip3 install PySide2 --user --ignore-installed

And that got me unstuck, although I'm struggling to understand how/why, and what to change in my package's dependency list, in order to fix this for everyone else. I suppose I could add a note to my installation instructions that says:

Now, do this: pip3 install PySide2 --user --ignore-installed, and don't ask why.

But, that seems rather inelegant.

Here are my sleuthing notes, so far:

Test Working Hypothesis: Pip installs something above and beyond what Conda installs when PySide2 is requested.

So, my working hypothesis as to the mystery above is this:

pip3 install PySide2 --user --ignore-installed put something extra into my base environment, which conda did not when it installed pyside2, as a consequence of being asked to install pybert.

Now, can I prove it?

Comparing dependency lists

First, let's just compare the respective dependency lists:

`conda`

(pybert-dev)
capnf@DESKTOP-G84ND7C MINGW64 ~
$ conda list pyside2
# packages in environment at C:\Users\capnf\anaconda3\envs\pybert-dev:
#
# Name                    Version                   Build  Channel
pyside2                   5.15.8           py39h56cbfbc_2    conda-forge

$ conda search --info -c conda-forge pyside2==5.15.8
{snip}
pyside2 5.15.8 py39h56cbfbc_2
-----------------------------
file name   : pyside2-5.15.8-py39h56cbfbc_2.conda
name        : pyside2
version     : 5.15.8
build       : py39h56cbfbc_2
build number: 2
size        : 8.0 MB
license     : LGPL-3.0-only
subdir      : win-64
url         : https://conda.anaconda.org/conda-forge/win-64/pyside2-5.15.8-py39h56cbfbc_2.conda
md5         : 880a9576f0b6e296c39c23e8df9d3af5
timestamp   : 2023-01-11 21:49:07 UTC
dependencies:
  - libclang13 >=15.0.6
  - libxml2 >=2.10.3,<2.11.0a0
  - libxslt >=1.1.37,<2.0a0
  - python >=3.9,<3.10.0a0
  - python_abi 3.9.* *_cp39
  - qt-main >=5.15.6,<5.16.0a0
  - ucrt >=10.0.20348.0
  - vc >=14.2,<15
  - vs2015_runtime >=14.29.30139

$ conda deactivate

(base)
capnf@DESKTOP-G84ND7C MINGW64 ~
$ conda list pyside2
# packages in environment at C:\Users\capnf\anaconda3:
#
# Name                    Version                   Build  Channel

(base)

`pip`

(base)
capnf@DESKTOP-G84ND7C MINGW64 ~
$ pip show pyside2
Name: PySide2
Version: 5.15.2.1
Summary: Python bindings for the Qt cross-platform application and UI framework
Home-page: https://www.pyside.org
Author: Qt for Python Team
Author-email: pyside@qt-project.org
License: LGPL
Location: c:\users\capnf\appdata\roaming\python\python39\site-packages
Requires: shiboken2
Required-by:

(base)

==> The dependency lists appear to be drastically different, pip requiring only shiboken2 and conda requiring several things, but not shiboken2.

Maybe, in the pip case, shiboken2 requires all that other stuff?
Nope:

(base)
capnf@DESKTOP-G84ND7C MINGW64 ~
$ pip show shiboken2
Name: shiboken2
Version: 5.15.2.1
Summary: Python / C++ bindings helper module
Home-page: https://www.pyside.org
Author: Qt for Python Team
Author-email: pyside@qt-project.org
License: LGPL
Location: c:\users\capnf\appdata\roaming\python\python39\site-packages
Requires:
Required-by: PySide2

Do I have shiboken2 installed in my conda environment(s)?
Nope:

(base)
capnf@DESKTOP-G84ND7C MINGW64 ~
$ conda list shiboken2
# packages in environment at C:\Users\capnf\anaconda3:
#
# Name                    Version                   Build  Channel

(base)
capnf@DESKTOP-G84ND7C MINGW64 ~
$ conda activate pybert-dev

(pybert-dev)
capnf@DESKTOP-G84ND7C MINGW64 ~
$ conda list shiboken2
# packages in environment at C:\Users\capnf\anaconda3\envs\pybert-dev:
#
# Name                    Version                   Build  Channel

(pybert-dev)

Is shiboken2 available from conda?
Nope:

(pybert-dev)
capnf@DESKTOP-G84ND7C MINGW64 ~
$ conda search --info -c conda-forge shiboken2
Loading channels: ...working... done

PackagesNotFoundError: The following packages are not available from current channels:

  - shiboken2

Current channels:

  - https://conda.anaconda.org/conda-forge/win-64
  - https://conda.anaconda.org/conda-forge/noarch
  - https://repo.anaconda.com/pkgs/main/win-64
  - https://repo.anaconda.com/pkgs/main/noarch
  - https://repo.anaconda.com/pkgs/r/win-64
  - https://repo.anaconda.com/pkgs/r/noarch
  - https://repo.anaconda.com/pkgs/msys2/win-64
  - https://repo.anaconda.com/pkgs/msys2/noarch

To search for alternate channels that may provide the conda package you're
looking for, navigate to

    https://anaconda.org

and use the search bar at the top of the page.

No match found for: shiboken2. Search: *shiboken2*
(pybert-dev)

Searching on anaconda.org yielded nothing.
Note: I do get several hits for just shiboken (no "2") and even one hit for: shiboken6.

corranwebster commented 1 year ago

Was on PTO, so apologies for not responding to this - you definitely have something missing from the PySide2 install if you can't import PySide2.QtCore (that's why we do a test import of this inside the pyface.qt infrastructure to make sure it's really there).

The most likely issue is that you need Shiboken 2.x - it is possible that it comes bundled with PySide2 in conda, but you can test by trying to import it (I can't remember whether it is shiboken or shiboken2).

The other possibility is a missing C++ dependency of the Qt build (eg. a shared library that it expects to be in the system but which is not) - this is a common issue on Linux, but I haven't seen it on Windows.

The only other thought I have is that your prompt mentions mingw64, but conda is almost certainly providing Visual Studio C++ runtime-compatible builds (it's been a long time but I recall code build with mingw being incompatible with VS builds; but that shouldn't be an issue in a pure conda environment).

I don't use conda myself (it's either EDM or pip for my work), so I'm not sure what to suggest beyond this.

capn-freako commented 1 year ago

The other possibility is a missing C++ dependency of the Qt build (eg. a shared library that it expects to be in the system but which is not) - this is a common issue on Linux, but I haven't seen it on Windows.

I think that was it, @corranwebster ; thanks!

enthought / pyface