pyblish / pyblish-qml

Pyblish QML frontend for Maya 2013+, Houdini 11+, Nuke 8+ and more
GNU Lesser General Public License v3.0
115 stars 44 forks source link

Windows 10 Blender throws ImportError: DLL load failed #368

Closed OlafHaag closed 3 years ago

OlafHaag commented 3 years ago

Hi! I created a proper Blender add-on with the abilitly to install the dependencies for pyblish-qml. Even though it's not public yet, I encounter the same issue I get with this implementation when trying to call pyblish_qml.show(): ImportError: DLL load failed Sometimes, the error is about permissions, other times it's that the specified module not found, depending on what I tried. It may be simillar to issues #301 and #313?

As it turned out, a solution to the problem was to copy the python3.dll from the Blender base installation folder to the /2.93/python/bin/ folder, where previously only python.exe, and python39.dll resided. More detailed error messages are in this issue. Obviously, I can't ask users to manually copy DLLs around to use the add-on. Besides asking the Blender devs to include the python3.dll in the python bin path as well and hope for it to be considered in a few years, I was hoping there is another, quicker, and user-friendly solution?

Windows 10 pyblish-base 1.8.8 pyblish-qml 1.11.5 PyQt5 5.15.4

mottosso commented 3 years ago

The problem sounds like you are attempting to use PyQt5 with Blender's build-in Python interpreter?

pyblish-qml does not depend on the host in any way, it does not require (nor recommend) that Blender has access to PyQt, and should open completely standalone without Blender, using an entirely external Python interpreter. Ideally one of the supported versions e.g. 3.7.

With that in mind, I would suggest:

  1. Confirm that you can run pyblish-qml standalone
  2. In Blender, register that standalone Python interpreter along with PyQt
  3. Standalone and Blender can share pyblish libraries, they are non-binary pure-Python

Here's a capture of a tested and supported combinations of Python and libraries.

OlafHaag commented 3 years ago

Thank you for your reply!

The problem sounds like you are attempting to use PyQt5 with Blender's build-in Python interpreter?

Yes. But the problem is that DLL are not found. I'm looking for a user-friendly setup. To be honest, having users install a separate python environment and manually pointing to that python executable and PyQt5 paths seems even worse than copying a DLL around.

pyblish-qml does not depend on the host in any way, it does not require (nor recommend) that Blender has access to PyQt

What are the downsides of using the python.exe that ships with Blender?

  1. Confirm that you can run pyblish-qml standalone

I created a conda environment with pyblish-base, pyblish-qml, and PyQt5. Running python -m pyblish_qml --demo works like expected.

  1. In Blender, register that standalone Python interpreter along with PyQt

Here I might get things wrong. In Blender I installed pyblish-base and pyblish-qml via pip. Otherwise, how would I register the python path and PyQt5 path? In Blender I did:

import pyblish.api
pyblish.api.register_host("blender")
import pyblish_qml.api
pyblish_qml.api.register_python_executable('C:\\Users\\user\\Anaconda3\\envs\\b3d293\\python.exe')
pyblish_qml.api.register_pyqt5("C:\\Users\\user\\Anaconda3\\envs\\b3d293\\Lib\\site-packages")
pyblish.api.register_gui("pyblish_qml")
pyblish_qml.show()

I still get:

Using Python @ 'C:\Users\user\Anaconda3\envs\b3d293\python.exe'
Using PyQt5 @ 'C:\Users\user\Anaconda3\envs\b3d293\Lib\site-packages'
Targets: default
Traceback (most recent call last):
File "C:\Users\user\Anaconda3\envs\b3d293\Lib\site-packages\pyblish_qml\vendor\Qt5.py", line 18, in <module>
from PySide2 import (
File "C:\Users\user\Anaconda3\envs\b3d293\Lib\site-packages\PySide2\__init__.py", line 51, in <module>
_setupQtDirectories()
File "C:\Users\user\Anaconda3\envs\b3d293\Lib\site-packages\PySide2\__init__.py", line 21, in _setupQtDirectories
import shiboken2
File "C:\Users\user\Anaconda3\envs\b3d293\Lib\site-packages\shiboken2\__init__.py", line 27, in <module>
from .shiboken2 import *
ImportError: DLL load failed while importing shiboken2: Das angegebene Modul wurde nicht gefunden.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\user\Anaconda3\envs\b3d293\lib\runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\Users\user\Anaconda3\envs\b3d293\lib\runpy.py", line 87, in _run_code
exec(code, run_globals)
File "C:\Users\user\Anaconda3\envs\b3d293\Lib\site-packages\pyblish_qml\__main__.py", line 6, in <module>
from . import app
File "C:\Users\user\Anaconda3\envs\b3d293\Lib\site-packages\pyblish_qml\app.py", line 12, in <module>
from . import util, compat, control, settings, ipc
File "C:\Users\user\Anaconda3\envs\b3d293\Lib\site-packages\pyblish_qml\util.py", line 10, in <module>
from .vendor.Qt5 import QtCore
File "C:\Users\user\Anaconda3\envs\b3d293\Lib\site-packages\pyblish_qml\vendor\Qt5.py", line 42, in <module>
from PyQt5 import (
ModuleNotFoundError: No module named 'PyQt5'
  1. Standalone and Blender can share pyblish libraries, they are non-binary pure-Python

Do you mean pyblish-base and pyblish-qml? Do I only register the plugin path in Blender?

mottosso commented 3 years ago

To be honest, having users install a separate python environment and manually pointing to that python executable and PyQt5 paths seems even worse than copying a DLL around.

Yes, the problem however is that the version of PyQt you use were built for the standalone Python from python.org, and you're attempting to use it with a different Python built specifically for Blender. In a nutshell, both Python and PyQt need to have been built with the same underlying compiler, e.g. Visual Studio 2019.

So your other option is shipping a PyQt built for Blender. One way or another, your Python and PyQt need to match for this to work.

For distribution purposes that does make a mess, but it's not unique to Blender. Anyone using Pyblish QML needs an external Python interpeter compatible with the version of PyQt they use. Most environments have that already for general pipeline purposes.

I created a conda environment with pyblish-base, pyblish-qml, and PyQt5. Running python -m pyblish_qml --demo works like expected.

Perfect, the Python used in this environment is what you'll need to use from Blender too.

C:\Users\user\Anaconda3\envs\b3d293\Lib\site-packages\shiboken2\__init__.py

It looks like your conda environment also has PySide2, and Qt5.py seems to pick that up instead. You can see here how it attempts to import bindings. Probably a poor choice even bothering to try PySide2, since pyblish-qml as of this writing only supports PyQt. Brainfart, we added PySide2 support not long ago. :D https://github.com/pyblish/pyblish-qml/pull/354

Aside from that, I also struggled to get on-top of this and also saw these errors when testing this myself. However, the cause is PyQt5 5.15. With 5.12 I'm seeing expected results.

import os
import sys

# Add to Blender
sys.path.insert(0, r"C:\Users\marcus\packages\pyblish_qml\1.11.5\python")
sys.path.insert(0, r"C:\Users\marcus\packages\pyblish_base\1.8.8\python")

# Add to Pyblish QML
os.environ["PYTHONPATH"] = r"C:\Users\marcus\packages\pyblish_qml\1.11.5\python"
os.environ["PYTHONPATH"] += r";C:\Users\marcus\packages\pyblish_base\1.8.8\python"

from pyblish_qml import api
api.register_python_executable(r"C:\Users\marcus\packages\python\3.7.3\platform-windows\arch-AMD64\app\python.exe")
api.register_pyqt5(r"C:\Users\marcus\packages\PyQt5\5.12.2\platform-windows\os-windows-10\python-3.7\python")
api.show()

image

The Add to sections is what I meant by both Blender and QML sharing pyblish libraries. You can add these to PYTHONPATH prior to launching Blender, and it will be automatically picked up by both. Anything added there is passed on to pyblish qml.

OlafHaag commented 3 years ago

Thanks for the information, Marcus!

In a nutshell, both Python and PyQt need to have been built with the same underlying compiler, e.g. Visual Studio 2019.

When I sys.version in Blender and in a standalone conda environment, I get: Blender:

'3.9.2 (default, Mar 1 2021, 08:18:55) [MSC v.1916 64 bit (AMD64)]'

Conda env:

'3.9.2 | packaged by conda-forge | (default, Feb 21 2021, 04:59:43) [MSC v.1916 64 bit (AMD64)]'

I can't quite tell the difference except for the date. To me it seems like the same compiler. So I'm still hopeful I could make it work with Blender's Python interpreter somehow.

The PyQt5 version 5.12.2 you reference can't be installed through pip (ERROR: No matching distribution found for PyQt5==5.12.2), nor through conda. Since that version there have been several DLL related issues reported on the webs. With newer versions I get:

File "C:\Users\user\Anaconda3\envs\b3d\Lib\site-packages\PyQt5\__init__.py", line 33, in find_qt
raise ImportError("unable to find Qt5Core.dll on PATH")
ImportError: unable to find Qt5Core.dll on PATH

Adding the dll directory to PATH as well, solves the issue for me:

import os
import sys

path = os.environ['PATH']
dll_dir = r"C:\Users\user\Anaconda3\envs\b3d\Library\bin"
if os.path.isfile(dll_dir + '\\Qt5Core_conda.dll'):
    path = dll_dir + ';' + path
    os.environ['PATH'] = path

# Add to Blender
sys.path.insert(0, r"C:\Users\user\Anaconda3\envs\b3d\Lib\site-packages")

# Add to Pyblish QML
os.environ["PYTHONPATH"] = r"C:\Users\user\Anaconda3\envs\b3d\Lib\site-packages"

from pyblish_qml import api
api.register_python_executable(r"C:\Users\user\Anaconda3\envs\b3d\python.exe")
api.register_pyqt5(r"C:\Users\user\Anaconda3\envs\b3d\Lib\site-packages")
api.show()

It's not the user-friendly solution I was hoping for and I doubt I can get a remote-team to go along with this intricate setup of a separate python environment. Yet, the provided workaround circumvents the DLL load failed error, so I'm closing this.

mottosso commented 3 years ago

The PyQt5 version 5.12.2 you reference can't be installed through pip

It doesn't exist for Python 3.9, it has been built for 3.7.

3.7 is the version used in Maya and all other DCCs, it is the one tagged by vfx platform for this generation. I would not recommend experimenting with other versions, and I would strongly recommend against using arbitrary DLLs that appear to work at first glance.

One option is to put Python 3.7 into a standalone folder, install PyQt5 into its local site-packages and ship that. Call it C:\Program Files\pyblish-qml-runtime and try not to think of it as a "separate Python environment"; it is the application files for Pyblish QML, much like how Maya or Blender or Discord or Sublime Text have application files.