Impossible to import a `pyd` which requires loading external DLLs despite them being in the same directory

[lighterowl]

It's not possible to load pyds which depend on other non-system DLLs, even if those DLLs are located in the same directory as the pyd. mingw-w64-x86_64-python version is 3.11.4-6, which includes the patches introduced in #144 .

A minimal example using cmake and pybind11 is here : https://github.com/xavery/pybind-with-dll . It's essentially a minified https://github.com/pybind/cmake_example but with an extra DLL in between.

Building this with said mingw-w64-x86_64-python gives :

$ mkdir build; cd build;
$ cmake ..
-- snip --
$ ninja
[4/4] Linking CXX shared module cmake_example.cp311-mingw_x86_64.pyd
$ python
Python 3.11.4 (main, Aug 22 2023, 21:54:38)  [GCC 13.2.0 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import cmake_example
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: DLL load failed while importing cmake_example: The specified module could not be found.

Setting PYTHONLEGACYWINDOWSDLLLOADING=1 works.

$ export PYTHONLEGACYWINDOWSDLLLOADING=1
$ python
Python 3.11.4 (main, Aug 22 2023, 21:54:38)  [GCC 13.2.0 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import cmake_example
>>> cmake_example.add(1,2)
3

This should still work without this variable being defined, though, as the flags passed to LoadLibraryEx in dynload_win.c include LOAD_LIBRARY_SEARCH_DLL_LOAD_DIR, which should add the originally loaded DLL's directory to the search path :

If this value is used, the directory that contains the DLL is temporarily added to the beginning of the list of directories that are searched for the DLL's dependencies. Directories in the standard search path are not searched. The lpFileName parameter must specify a fully qualified path.

I presume that wpathname might not be a fully qualified path at the time LoadLibraryEx is called but I have not checked this myself.

If you consider this working as desired, please consider removing LOAD_LIBRARY_SEARCH_DLL_LOAD_DIR from the flags passed to LoadLibraryEx as it's misleading and hints that it should be possible to load the dependencies from the directory even without calling AddDllDirectory (or its Python wrapper) explicitly.

Thanks.

[lazka]

Thanks

[MehdiChinoune]

I think I hit the same issue when I tried to update pyside6.

[lazka]

I presume that wpathname might not be a fully qualified path at the time LoadLibraryEx is called but I have not checked this myself.

I think you are correct, as MSYSTEM= python3 -c "import cmake_example" works. Likely LoadLibraryEx only considers paths using backslashes as fully qualified (??). So we need to normalize the path separators when building the path to load.

Thanks for the nice reproducer.

[lazka]

@xavery the fix is in the MSYS2 repo now