Windows 0x7e error cffi fails to load dll library

[amrosado]

python version: 3.11.9 pyvips: 2.2.3 cffi: 1.16.0 libvips: 8.15.3

Pyvips fails to load dll library despite the following code:

vips_bin_dir = os.path.abspath(os.path.join('..', 'ImageTools', 'vips-dev-815-win', 'bin'))
add_dll_dir = getattr(os, 'add_dll_directory', None)
if callable(add_dll_dir):
    add_dll_dir(vips_bin_dir)
else:
    os.environ['PATH'] = os.pathsep.join((vips_bin_dir, os.environ['PATH']))

cffi version installed by pip cannot load library using ffi.dlopen. Additionally, importing pyvips attempts to use cffi to load library, but this fails regardless of dll version with and without using ffi/static dll. This fails even when using cffi outside of the scope of pyvips. Upgrading cffi to 1.17.0rc1 seems to enable using dlopen to open the library. Despite being able to use cffi to independently load the library import pyvips still fails when attempting to load_library with a path that is not absolute. The following code enables importing pyvips in my environment.

vips_bin_dir = os.path.abspath(os.path.join('..', 'ImageTools', 'vips-dev-815-win', 'bin'))
ffi.dlopen(os.path.join(vips_bin_dir, 'libvips-42.dll'))
add_dll_dir = getattr(os, 'add_dll_directory', None)
os.environ['PATH'] = os.pathsep.join((vips_bin_dir, os.environ['PATH']))

if callable(add_dll_dir):
    add_dll_dir(vips_bin_dir)
else:
    os.environ['PATH'] = os.pathsep.join((vips_bin_dir, os.environ['PATH']))

In python 3.11, I decided to use the following code to successfully import pyvips:

vips_bin_dir = os.path.abspath(os.path.join('..', 'DsaLargeImageTools', 'vips-dev-815-win', 'bin'))
add_dll_dir = getattr(os, 'add_dll_directory', None)
os.environ['PATH'] = os.pathsep.join((vips_bin_dir, os.environ['PATH']))

I am able to find the library and load the library independently with ctypes. This seems to also not work with libvips 8.13 which is described in the current documentation.

It would be beneficial I think to suggest to windows users it is okay to not dynamically add the dll directory at runtime despite this being the preferred way to do load dlls using the winapi.

[jcupitt]

Hello @amrosado,

You're right, I had the same error using add_dll_dir(), I wonder what's changed?

I saw:

jcupi@DESKTOP-HGI6HBR MINGW64 /c/Python311
$ python3.11.exe -m pip install pyvips
Collecting pyvips
  Downloading pyvips-2.2.3.tar.gz (56 kB)
     ---------------------------------------- 56.6/56.6 kB 3.1 MB/s eta 0:00:00
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Installing backend dependencies: started
  Installing backend dependencies: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Collecting cffi>=1.0.0 (from pyvips)
  Using cached cffi-1.16.0-cp311-cp311-win_amd64.whl (181 kB)
Collecting pycparser (from cffi>=1.0.0->pyvips)
  Using cached pycparser-2.22-py3-none-any.whl (117 kB)
Building wheels for collected packages: pyvips
  Building wheel for pyvips (pyproject.toml): started
  Building wheel for pyvips (pyproject.toml): finished with status 'done'
  Created wheel for pyvips: filename=pyvips-2.2.3-py2.py3-none-any.whl size=54890 sha256=736a49fa0aaf781f7d1b211253d956565ec348f539bf93cb73916dfbb4a8d2e4
  Stored in directory: c:\users\jcupi\appdata\local\pip\cache\wheels\fc\2c\3a\120103ac3f113407daed5416c5386cd13172c92f68ee9f7208
Successfully built pyvips
Installing collected packages: pycparser, cffi, pyvips
Successfully installed cffi-1.16.0 pycparser-2.22 pyvips-2.2.3

Then:

Python 3.11.4 (tags/v3.11.4:d2340ef, Jun  7 2023, 05:45:37) [MSC v.1934 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> vipsbin = r'f:\vips-dev-8.15\bin'
>>> add_dll_dir = getattr(os, 'add_dll_directory', None)
>>> add_dll_dir(vipsbin)
<AddedDllDirectory('f:\\vips-dev-8.15\\bin')>
>>> import pyvips
Traceback (most recent call last):
  File "C:\Python311\Lib\site-packages\pyvips\__init__.py", line 19, in <module>
    import _libvips
ModuleNotFoundError: No module named '_libvips'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Python311\Lib\site-packages\pyvips\__init__.py", line 57, in <module>
    vips_lib = ffi.dlopen('libvips-42.dll')
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python311\Lib\site-packages\cffi\api.py", line 150, in dlopen
    lib, function_cache = _make_ffi_library(self, name, flags)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python311\Lib\site-packages\cffi\api.py", line 832, in _make_ffi_library
    backendlib = _load_backend_lib(backend, libname, flags)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Python311\Lib\site-packages\cffi\api.py", line 827, in _load_backend_lib
    raise OSError(msg)
OSError: cannot load library 'libvips-42.dll': error 0x7e.  Additionally, ctypes.util.find_library() did not manage to locate a library called 'libvips-42.dll'
>>>

However this works:

Python 3.11.4 (tags/v3.11.4:d2340ef, Jun  7 2023, 05:45:37) [MSC v.1934 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> vipsbin = r'f:\vips-dev-8.15\bin'
>>> os.environ['PATH'] = os.pathsep.join((vipsbin, os.environ['PATH']))
>>> import pyvips
>>> x = pyvips.Image.black(100,100)
>>> x
<pyvips.Image 100x100 uchar, 1 bands, multiband>
>>>

The add_dll_dir() method was certainly working last year :(

I'll add a note to the README, thank you for alerting me to this problem.

[jcupitt]

Thinking again, before making a recommendation, it would be good to find out why the add_dll_dir() method sometimes fails. Is it linked to the cffi version?

[amrosado]

I was trying to figure that out myself, because I would prefer the safer way of loading the dll. Some things I notice when looking at the code were that cffi's ffi.dlopen is given 'libvips-42.dll' which I believe works when the path is configured, but could get an absolute path to the dll. I also noticed in cffi's _load_backend_lib that ctypes.util.find_library is given the dll file name as the name variable, but the documentation reads that it should be a name of the binary without an extension (dll, so).