PySide2 5.15.2.1: ImportError: DLL load failed while importing shiboken2: The specified module could not be found.

[LexiconCode]

This error occurred in python 3.10.2 and 3.8.9 32-bit. Simply load '_test.py' with a the import from PySide2 import QtCore to reproduce. The only happens with Natlink and not other engines or dragonfly text engine.

shiboken2 does exist in: "C:\Users\MainLaptop\AppData\Local\Programs\Python\Python310-32\Lib\site-packages\shiboken2\shiboken2.pyd"

Natlink Version: 5.0.0
Python Version: 3.10.2 (tags/v3.10.2:a58ebcc, Jan 17 2022, 13:59:55) [MSC v.1929 32 bit (Intel)]
NatLink: LOADED!
PySide2/__init__.py: Unable to import shiboken2 from C:\Users\MainLaptop\Documents\Caster, C:\Program Files (x86)\Natlink\site-packages\natlink\python310.zip, C:\Program Files (x86)\Natlink\site-packages, C:\Users\MainLaptop\AppData\Local\Programs\Python\Python310-32\Lib, C:\Users\MainLaptop\AppData\Local\Programs\Python\Python310-32\DLLs, C:\Program Files (x86)\Nuance\NaturallySpeaking13\Program, C:\Users\MainLaptop\AppData\Local\Programs\Python\Python310-32, C:\Users\MainLaptop\AppData\Local\Programs\Python\Python310-32\lib\site-packages, C:\Users\MainLaptop\AppData\Local\Programs\Python\Python310-32\lib\site-packages\win32, C:\Users\MainLaptop\AppData\Local\Programs\Python\Python310-32\lib\site-packages\win32\lib, C:\Users\MainLaptop\AppData\Local\Programs\Python\Python310-32\lib\site-packages\Pythonwin
Traceback (most recent call last):
  File "C:\Program Files (x86)\Natlink\site-packages\natlink\loader.py", line 138, in load_or_reload_module
    module = self._import_module_from_path(mod_path)
  File "C:\Program Files (x86)\Natlink\site-packages\natlink\loader.py", line 111, in _import_module_from_path
    loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "C:\Users\MainLaptop\Documents\Caster\_p.py", line 1, in <module>
    from PySide2 import QtCore
  File "C:\Users\MainLaptop\AppData\Local\Programs\Python\Python310-32\lib\site-packages\PySide2\__init__.py", line 107, in <module>
    _setupQtDirectories()
  File "C:\Users\MainLaptop\AppData\Local\Programs\Python\Python310-32\lib\site-packages\PySide2\__init__.py", line 58, in _setupQtDirectories
    import shiboken2
  File "C:\Users\MainLaptop\AppData\Local\Programs\Python\Python310-32\lib\site-packages\shiboken2\__init__.py", line 27, in <module>
    from .shiboken2 import *
ImportError: DLL load failed while importing shiboken2: The specified module could not be found.
Traceback (most recent call last):
  File "C:\Program Files (x86)\Natlink\site-packages\natlink\loader.py", line 138, in load_or_reload_module
    module = self._import_module_from_path(mod_path)
  File "C:\Program Files (x86)\Natlink\site-packages\natlink\loader.py", line 111, in _import_module_from_path
    loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "C:\Users\MainLaptop\Documents\Caster\_p.py", line 1, in <module>
    from PySide2 import QtCore
  File "C:\Users\MainLaptop\AppData\Local\Programs\Python\Python310-32\lib\site-packages\PySide2\__init__.py", line 107, in <module>
    _setupQtDirectories()
  File "C:\Users\MainLaptop\AppData\Local\Programs\Python\Python310-32\lib\site-packages\PySide2\__init__.py", line 58, in _setupQtDirectories
    import shiboken2
  File "C:\Users\MainLaptop\AppData\Local\Programs\Python\Python310-32\lib\site-packages\shiboken2\__init__.py", line 27, in <module>
    from .shiboken2 import *
ImportError: DLL load failed while importing shiboken2: The specified module could not be found.
Attempting to load duplicate module: C:\Users\MainLaptop\Documents\_test.py)

Adding shiboken2.pyd to path sys.path.append leads to no change.

[LexiconCode]

Natlink in debug mode

Natlink Version: 5.0.0
Python Version: 3.10.2 (tags/v3.10.2:a58ebcc, Jan 17 2022, 13:59:55) [MSC v.1929 32 bit (Intel)]
NatLink: LOADED!
set log level to: DEBUG
starting natlink loader from config file:
    "C:\Users\MainLaptop\natlink.ini"
triggering load/reload process
loading module: _test
PySide2/__init__.py: Unable to import shiboken2 from C:\Users\MainLaptop\Documents\Caster, C:\Program Files (x86)\Natlink\site-packages\natlink\python310.zip, C:\Program Files (x86)\Natlink\site-packages, C:\Users\MainLaptop\AppData\Local\Programs\Python\Python310-32\Lib, C:\Users\MainLaptop\AppData\Local\Programs\Python\Python310-32\DLLs, C:\Program Files (x86)\Nuance\NaturallySpeaking13\Program, C:\Users\MainLaptop\AppData\Local\Programs\Python\Python310-32, C:\Users\MainLaptop\AppData\Local\Programs\Python\Python310-32\lib\site-packages, C:\Users\MainLaptop\AppData\Local\Programs\Python\Python310-32\lib\site-packages\win32, C:\Users\MainLaptop\AppData\Local\Programs\Python\Python310-32\lib\site-packages\win32\lib, C:\Users\MainLaptop\AppData\Local\Programs\Python\Python310-32\lib\site-packages\Pythonwin
Traceback (most recent call last):
  File "C:\Program Files (x86)\Natlink\site-packages\natlink\loader.py", line 138, in load_or_reload_module
    module = self._import_module_from_path(mod_path)
  File "C:\Program Files (x86)\Natlink\site-packages\natlink\loader.py", line 111, in _import_module_from_path
    loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "C:\Users\MainLaptop\Documents\Caster\_test.py", line 1, in <module>
    from PySide2 import QtCore
  File "C:\Users\MainLaptop\AppData\Local\Programs\Python\Python310-32\lib\site-packages\PySide2\__init__.py", line 107, in <module>
    _setupQtDirectories()
  File "C:\Users\MainLaptop\AppData\Local\Programs\Python\Python310-32\lib\site-packages\PySide2\__init__.py", line 58, in _setupQtDirectories
    import shiboken2
  File "C:\Users\MainLaptop\AppData\Local\Programs\Python\Python310-32\lib\site-packages\shiboken2\__init__.py", line 27, in <module>
    from .shiboken2 import *
ImportError: DLL load failed while importing shiboken2: The specified module could not be found.
Traceback (most recent call last):
  File "C:\Program Files (x86)\Natlink\site-packages\natlink\loader.py", line 138, in load_or_reload_module
    module = self._import_module_from_path(mod_path)
  File "C:\Program Files (x86)\Natlink\site-packages\natlink\loader.py", line 111, in _import_module_from_path
    loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "C:\Users\MainLaptop\Documents\Caster\_test.py", line 1, in <module>
    from PySide2 import QtCore
  File "C:\Users\MainLaptop\AppData\Local\Programs\Python\Python310-32\lib\site-packages\PySide2\__init__.py", line 107, in <module>
    _setupQtDirectories()
  File "C:\Users\MainLaptop\AppData\Local\Programs\Python\Python310-32\lib\site-packages\PySide2\__init__.py", line 58, in _setupQtDirectories
    import shiboken2
  File "C:\Users\MainLaptop\AppData\Local\Programs\Python\Python310-32\lib\site-packages\shiboken2\__init__.py", line 27, in <module>
    from .shiboken2 import *
ImportError: DLL load failed while importing shiboken2: The specified module could not be found.
second round, load new grammar files: {WindowsPath('C:/Users/MainLaptop/Documents/Caster/_test.py')}
new module in second round: C:\Users\MainLaptop\Documents\Caster\_test.py
Attempting to load duplicate module: C:\Users\MainLaptop\Documents\Caster\_test.py)

[quintijn]

I tried one line extra in loader.py, according to manual pages of importlib. Cannot test right now, because my ssh connection is broken with a new windows update. We will discuss this, LexiconCode, tuesday! Greetings, Quintijn

[fusentasticus]

Maybe this (https://stackoverflow.com/questions/41365446/how-to-resolve-importerror-dll-load-failed-on-python)???

If anyone comes across this issue in Python > 3.8 with Windows, dll's are only loaded from trusted locations https://docs.python.org/3/whatsnew/3.8.html#ctypes This can be fixed by adding the dll path using os.add_dll_directory("PATH_TO_DLL")

[LexiconCode]

What's interesting here is Caster already utilizes PySide2 which up until now has been fine. The module doesn't mind being loaded later in Casters code but doesn't like being imported directly in natlink or early. In caster PySide2 is not even imported directly but is launched as a separate process with communication via pythons multiprocessing client and listener.

[LexiconCode]

I tried one line extra in loader.py, according to manual pages of importlib. Cannot test right now, because my ssh connection is broken with a new windows update. We will discuss this, LexiconCode, tuesday! Greetings, Quintijn

Unfortunately no change

[LexiconCode]

os.add_dll_directory("PATH_TO_DLL")

No dice unfortunately

import os
os.add_dll_directory(r"C:\Users\Main\AppData\Local\Programs\Python\Python38-32\Lib\site-packages\shiboken2")
import PySide2

[fusentasticus]

Check whether by chance fix to https://github.com/dictation-toolbox/natlink/issues/86 helps (I doubt it). Then try to find out whether it's really that the dll can't be found and that this is really not a dll load of an actually existing file that fails. Check that there is only one shiboken2 directory on sys.path and list its contents of globbed stuff (.py and .pyd and .dll) - or replace

from .shiboken2 import *from .shiboken2 import *

with a list of specific imports - that is replace the glob (*) with individual imports.

[LexiconCode]

check whether by chance fix to #86 helps (I doubt it)

Unfortunately not

with a list of specific imports - that is replace the glob (*) with individual imports.

it looks like it's importing files but not *.py

PS C:\Users\Main> .'C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.31.31103\bin\Hostx86\x86\dumpbin.exe' /DEPENDENTS "C:\Users\Main\AppData\Roaming\Python\Python310-32\site-packages\shiboken2\shiboken2.pyd"
Microsoft (R) COFF/PE Dumper Version 14.31.31104.0
Copyright (C) Microsoft Corporation.  All rights reserved.

Dump of file C:\Users\Main\AppData\Roaming\Python\Python310-32\site-packages\shiboken2\shiboken2.pyd

File Type: DLL

  Image has the following dependencies:

    shiboken2.abi3.dll
    python3.dll
    VCRUNTIME140.dll
    api-ms-win-crt-runtime-l1-1-0.dll
    api-ms-win-crt-heap-l1-1-0.dll
    KERNEL32.dll

  Summary

        1000 .data
        2000 .rdata
        1000 .reloc
        1000 .rsrc
        2000 .text

[LexiconCode]

Looks like I'm going to have to go all the way and install the Windows development environment

https://docs.microsoft.com/en-us/archive/blogs/junfeng/debugging-loadlibrary-failures https://developer.microsoft.com/en-us/windows/downloads/windows-sdk/

."C:\Program Files (x86)\Windows Kits\10\Debuggers\x86\gflags.exe" -i "C:\Program Files (x86)\Python310-32\python.exe" +sls
."C:\Program Files (x86)\Python310-32\python.exe" -c 'import shiboken2'

[LexiconCode]

Another interesting tidbit here regarding the Python environment

Installing through Python through the natlink installer in looking at were Python packages are being stored.

PS C:\Users\Main> ."C:\Program Files (x86)\Python310-32\python.exe" -m pip install shiboken2
Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: shiboken2 in c:\users\main\appdata\roaming\python\python310-32\site-packages (5.15.2.1)
PS C:\Users\Main>

[dougransom]

i am not following this closely, so this is just something to consider. we ran into problems importing natlink.dll when there was a folder or module named natlink. so if you have a shiboken2 python module or file and a shiboken2.pyd you probably will run into troubles.

[LexiconCode]

i am not following this closely, so this is just something to consider. we ran into problems importing natlink.dll when there was a folder or module named natlink. so if you have a shiboken2 python module or file and a shiboken2.pyd you probably will run into troubles.

Yes I'm aware of this but what I haven't been able to figure out is why this is explicitly happening only when loading through Natlink (assuming it's underlying issue) as everything works through the standard Python environment. Being able to easily attach a debugger before Nalink tries to load grammars would be a help. https://github.com/dictation-toolbox/natlink/issues/91

[dougransom]

Which folder for site-packages would depend on whether you installed python for all users.

From: LexiconCode @.> Sent: Friday, March 11, 2022 8:20 AM To: dictation-toolbox/natlink @.> Cc: Subscribed @.***> Subject: Re: [dictation-toolbox/natlink] PySide2 5.15.2.1: ImportError: DLL load failed while importing shiboken2: The specified module could not be found. (Issue #89)

another interesting tidbit here

Installing through Python through the natlink installer in looking at were Python packages are being stored.

It's interesting to see that C:\Program Files (x86)\Python310-32\Lib\site-packages is not being used but c:\users\main\appdata\roaming\python\python310-32\site-packages

- Reply to this email directly, view it on GitHubhttps://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fdictation-toolbox%2Fnatlink%2Fissues%2F89%23issuecomment-1065267511&data=04%7C01%7C%7C58415777f7924ac632c608da037af3a6%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637826123864537039%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=Zww3%2Bm0M75Cwtry7NE76Gj4sVjzv0qWlxUDLeNpXfG0%3D&reserved=0, or unsubscribehttps://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAA262UW7BGO3XAUVEYZXWVLU7NXBTANCNFSM5OK7CLRA&data=04%7C01%7C%7C58415777f7924ac632c608da037af3a6%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637826123864537039%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=4XomjUu2S3TIl4dgaECKvwZjQWFrWL%2Ft1Sv7vlS4ps4%3D&reserved=0. Triage notifications on the go with GitHub Mobile for iOShttps://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapps.apple.com%2Fapp%2Fapple-store%2Fid1477376905%3Fct%3Dnotification-email%26mt%3D8%26pt%3D524675&data=04%7C01%7C%7C58415777f7924ac632c608da037af3a6%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637826123864537039%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=2SDG6lPN1ysZUVK9EgtX%2BBlq0u5%2FhPJajBOrmF9h56o%3D&reserved=0 or Androidhttps://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fplay.google.com%2Fstore%2Fapps%2Fdetails%3Fid%3Dcom.github.android%26referrer%3Dutm_campaign%253Dnotification-email%2526utm_medium%253Demail%2526utm_source%253Dgithub&data=04%7C01%7C%7C58415777f7924ac632c608da037af3a6%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637826123864537039%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=t7d1Eewt0WW9aaaukEg%2BRH9pM05vHZlbT%2BsoaVQ3vRY%3D&reserved=0. You are receiving this because you are subscribed to this thread.Message ID: @.**@.>>

[LexiconCode]

Same issues with PyQt5

_import_module_from_path
    loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "C:\Users\MainLaptop\Documents\_x.py", line 1, in <module>
    from PyQt5 import QtWidgets, QtCore, QtGui
ImportError: DLL load failed while importing QtWidgets: The specified module could not be found.
Traceback (most recent call last):
  File "C:\Users\MainLaptop\AppData\Local\Programs\Python\Python310-32\lib\site-packages\natlinkcore\loader.py", line 305, in load_or_reload_module
    module = self._import_module_from_path(mod_path)
  File "C:\Users\MainLaptop\AppData\Local\Programs\Python\Python310-32\lib\site-packages\natlinkcore\loader.py", line 261, in _import_module_from_path
    loader.exec_module(module)
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "C:\Users\MainLaptop\Documents\_x.py", line 1, in <module>
    from PyQt5 import QtWidgets, QtCore, QtGui
ImportError: DLL load failed while importing QtWidgets: The specified module could not be found.

[dougransom]

@LexiconCode Can you please post the output of pip list?

[LexiconCode]

pip list

Package Version Editable project location

---

appdirs 1.4.4 attrdict 2.0.1 CacheControl 0.12.11 cachy 0.3.0 certifi 2021.10.8 charset-normalizer 2.0.12 cleo 0.8.1 clikit 0.6.2 comtypes 1.1.11 crashtest 0.3.1 debugpy 1.6.2 decorator 5.1.1 distlib 0.3.5 docutils 0.19 dragonfly2 0.34.0 dtactions 1.5.2 C:\Users\MainLaptop\Documents\code\dtactions filelock 3.7.1 flit 3.7.1 flit_core 3.7.1 future 0.18.2 html5lib 1.1 idna 3.3 json-rpc 1.13.0 keyring 23.7.0 lark-parser 0.8.9 lockfile 0.12.2 mock 4.0.3 msgpack 1.0.4 natlinkcore 5.2.2 C:\Users\MainLaptop\Documents\natlink packaging 20.9 pastel 0.2.1 pexpect 4.8.0 Pillow 9.0.1 pip 22.2.2 pkginfo 1.8.3 platformdirs 2.5.2 poetry 1.1.14 poetry-core 1.0.8 ptyprocess 0.7.0 pydebugstring 0.0.0.2 pyenvutils 1.0.0.2 pylev 1.4.0 pyparsing 3.0.7 pyperclip 1.8.2 PyQt5 5.15.7 PyQt5-Qt5 5.15.2 PyQt5-sip 12.11.0 PySimpleGUI 4.60.3 pyvda 0.0.8 pywin32 303 pywin32-ctypes 0.2.0 regex 2022.1.18 requests 2.27.1 requests-toolbelt 0.9.1 scandir 1.10.0 setuptools 63.2.0 shellingham 1.5.0 six 1.16.0 tomli 2.0.1 tomli_w 1.0.0 tomlkit 0.10.0 urllib3 1.26.8 virtualenv 20.16.3 webencodings 0.5.1 Werkzeug 2.0.3

[dougransom]

This reminds me of the troubles I had when I attempted to have natlink use a virtual environment.

Try setting the global Windows path environment variable control panel to include the necessary DLLs such as `

shiboken2.abi3.dll
python3.dll
VCRUNTIME140.dll
api-ms-win-crt-runtime-l1-1-0.dll
api-ms-win-crt-heap-l1-1-0.dll
KERNEL32.dll`