Cannot access from my virtual environment

[FaxMeBeer]

I followed the docs for installation to Windows 10, and as long as I'm not in my Django project's virtual environment, everything works fine. When I activate my virtual environment, I get the

OSError: no library called "cairo" was found
cannot load library 'C:\Users\####\####\Lib\site-packages\GTK3-Runtime Win64\bin\libcairo-2.dll': error 0x7e
cannot load library 'libcairo.so.2': error 0x7e
cannot load library 'libcairo.2.dylib': error 0x7e
cannot load library 'libcairo-2.dll': error 0x7e

When I check the

WHERE zlib1.dll

From the environment, the path is found and correct. And the same is true for WHERE libcairo-2.dll.

Since the path is clearly visible within the environment, I don't understand why I'm getting the error when I run the test command:

python -m weasyprint http://weasyprint.org weasyprint.pdf

From my environment, but not from the standard command prompt.

[Tontyna]

What is the output of

python -c "import os; print(*os.environ['PATH'].split(os.pathsep), sep='\n')"

when you run it in your Django environment?

[FaxMeBeer]

The output is:

C:\Program Files\GTK3-Runtime Win64\bin
c:\Users\drcar\collect_it\Scripts
C:\Program Files (x86)\Common Files\Oracle\Java\javapath
C:\WINDOWS\system32
C:\WINDOWS
C:\WINDOWS\System32\Wbem
C:\WINDOWS\System32\WindowsPowerShell\v1.0\
C:\WINDOWS\System32\OpenSSH\
C:\Users\drcar\collect_it\Git\cmd
C:\Program Files\GTK3-Runtime Win64\bin
C:\Users\drcar\AppData\Local\Programs\Python\Python38\Scripts\
C:\Users\drcar\AppData\Local\Programs\Python\Python38\
C:\Users\drcar\AppData\Local\Microsoft\WindowsApps
C:\Program Files\JetBrains\PyCharm Community Edition 2020.1.1\bin
C:\Users\drcar\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\Scripts
C:\Users\drcar\AppData\Local\Programs\Microsoft VS Code\bin
C:\Users\drcar\AppData\Local\atom\bin
C:\Users\drcar\AppData\Local\Microsoft\WindowsApps

[Tontyna]

None of the paths in front of your GTK3-Runtime Win64\bin path looks especially suspicious to me, but who knows...

What happens if you execute the following Python script?

import os

# move it to the front
GTK_PATH = r"C:\Program Files\GTK3-Runtime Win64\bin"

os.environ['PATH'] =  GTK_PATH + os.pathsep + os.environ.get('PATH','')

from weasyprint import HTML
HTML('http://weasyprint.org/').write_pdf('weasyprint-website.pdf')

[FaxMeBeer]

Here is what I get:

>>> import os
>>> GTK_PATH = r"C:\Program Files\GTK3-Runtime Win64\bin"
>>> os.environ['PATH'] =  GTK_PATH + os.pathsep + os.environ.get('PATH','')
>>> from weasyprint import HTML
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "c:\Users\drcar\collect_it\lib\site-packages\weasyprint\__init__.py", line 469, in <module>
    from .css import preprocess_stylesheet  # noqa isort:skip
  File "c:\Users\drcar\collect_it\lib\site-packages\weasyprint\css\__init__.py", line 27, in <module>
    from . import computed_values, counters, media_queries
  File "c:\Users\drcar\collect_it\lib\site-packages\weasyprint\css\computed_values.py", line 15, in <module>
    from .. import text
  File "c:\Users\drcar\collect_it\lib\site-packages\weasyprint\text.py", line 11, in <module>
    import cairocffi as cairo
  File "c:\Users\drcar\collect_it\lib\site-packages\cairocffi\__init__.py", line 48, in <module>
    cairo = dlopen(
  File "c:\Users\drcar\collect_it\lib\site-packages\cairocffi\__init__.py", line 45, in dlopen
    raise OSError(error_message)  # pragma: no cover
OSError: no library called "cairo" was found
cannot load library 'C:\Program Files\GTK3-Runtime Win64\bin\libcairo-2.dll': error 0x7e
cannot load library 'libcairo.so.2': error 0x7e
cannot load library 'libcairo.2.dylib': error 0x7e
cannot load library 'libcairo-2.dll': error 0x7e

[Tontyna]

Now I'm really puzzled about the various paths -- in my previous comment I erroneously typed (copy-pasted) C:\Program Files\GTK3-Runtime Win64\bin instead of C:\Users\drcar\collect_it\Lib\site-packages\GTK3-Runtime Win64\bin... your edited output of os.environ['PATH'] doesn't look like the one of your Django environment.

What happens if you execute (in your Django environment):

SET GTKPATH=C:\Users\drcar\collect_it\Lib\site-packages\GTK3-Runtime Win64\bin
SET "PATH=%GTKPATH%;%PATH%"
python -m weasyprint http://weasyprint.org weasyprint.pdf

[FaxMeBeer]

Maybe I'm too much of a noob for this :). I'm sorry.

I do get the same error:

(collect_it) C:\Users\drcar\collect_it\collect_it>SET GTKPATH=C:\Users\drcar\collect_it\Lib\site-packages\GTK3-Runtime Win64\bin

(collect_it) C:\Users\drcar\collect_it\collect_it>SET "PATH=%GTKPATH%;%PATH%"

(collect_it) C:\Users\drcar\collect_it\collect_it>python -m weasyprint http://weasyprint.org weasyprint.pdf
Traceback (most recent call last):
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.8_3.8.2288.0_x64__qbz5n2kfra8p0\lib\runpy.py", line 185, in _run_module_as_main
    mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.8_3.8.2288.0_x64__qbz5n2kfra8p0\lib\runpy.py", line 144, in _get_module_details     
    return _get_module_details(pkg_main_name, error)
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.8_3.8.2288.0_x64__qbz5n2kfra8p0\lib\runpy.py", line 111, in _get_module_details     
    __import__(pkg_name)
  File "c:\Users\drcar\collect_it\lib\site-packages\weasyprint\__init__.py", line 469, in <module>
    from .css import preprocess_stylesheet  # noqa isort:skip
  File "c:\Users\drcar\collect_it\lib\site-packages\weasyprint\css\__init__.py", line 27, in <module>
    from . import computed_values, counters, media_queries
  File "c:\Users\drcar\collect_it\lib\site-packages\weasyprint\css\computed_values.py", line 15, in <module>
    from .. import text
  File "c:\Users\drcar\collect_it\lib\site-packages\weasyprint\text.py", line 11, in <module>
    import cairocffi as cairo
  File "c:\Users\drcar\collect_it\lib\site-packages\cairocffi\__init__.py", line 48, in <module>
    cairo = dlopen(
  File "c:\Users\drcar\collect_it\lib\site-packages\cairocffi\__init__.py", line 45, in dlopen
    raise OSError(error_message)  # pragma: no cover
OSError: no library called "cairo" was found
cannot load library 'C:\Program Files\GTK3-Runtime Win64\bin\libcairo-2.dll': error 0x7e
cannot load library 'libcairo.so.2': error 0x7e
cannot load library 'libcairo.2.dylib': error 0x7e
cannot load library 'libcairo-2.dll': error 0x7e

[Tontyna]

I didn't research in detail how they actually resolve the DLL paths, but in Python 3.8 they obviously changed it:

[...] DLLs loaded with ctypes on Windows are now resolved more securely. Only the system paths, the directory containing the DLL or PYD file, and directories added with add_dll_directory() are searched for load-time dependencies. Specifically, PATH and the current working directory are no longer used, and modifications to these will no longer have any effect on normal DLL resolution.

Maybe that's the reason why the proven way of loading Cairo fails. Maybe add_dll_directory can help.

But where is your GTK3-Runtime? Do you have two of them?

• in C:\Users\drcar\collect_it\Lib\site-packages\GTK3-Runtime Win64\bin
• in C:\Program Files\GTK3-Runtime Win64\bin

[FaxMeBeer]

On the duplicate GTK3-Runtime files, I installed it once into C:\Users\drcar\collect_it\Lib\site-packages\GTK3-Runtime Win64\bin at a suggestion from Reddit (to put it in the same director as my virtual environment). When that didn't work, I re-installed it to C:\Program Files\GTK3-Runtime Win64\bin because the Reddit suggestion didn't work, and the Program Files option matched the WeasyPrint documentation.

In my file explorer, the one in site-packages\GTK3-Runtime is not visible, so I believe it was appropriately uninstalled when I did the install to the Program Files location, though I don't know why it's still showing up when I run `python -c "import os; print(*os.environ['PATH'].split(os.pathsep), sep='\n')".

I will Google around with add_dll_directory and see if I can figure that out. I really appreciate you being patient and helpful with me. Thank you.

[FaxMeBeer]

I have tried a couple of other things after searching for add_dll_directory. I found a Stack Overflow post ('django OSError: no library called "cairo" was found on windows" by Mhmoud Amsha. One answer said that installing GTK+ didn't fix the problem for them, so they used UniConverter2.0, and added the dll sub-directory to the advanced settings, so I did that, and received the same error.

In that same thread, there is a 0 rated answer that recommends adding this script:

import os

def set_dll_search_path():
   # Python 3.8 no longer searches for DLLs in PATH, so we have to add
   # everything in PATH manually. Note that unlike PATH add_dll_directory
   # has no defined order, so if there are two cairo DLLs in PATH we
   # might get a random one.
   if os.name != "nt" or not hasattr(os, "add_dll_directory"):
       return
   for p in os.environ.get("PATH", "").split(os.pathsep):
       try:
           os.add_dll_directory(p)
       except OSError:
           pass

set_dll_search_path()

But I honestly don't know where I'm supposed to do that at, and since nobody has up-voted that answer, I'm guessing it may not be a valid fix anyway.

[Tontyna]

Looks like the function on stackoverflow tries to restore the pre-3.8 behaviour by adding all paths from os.environ['PATH'] (i.e. the PATH) to the dll search directories.

To check whether add_dll_directory saves your day you can use a simplified Python script:

import os
GTK_PATH = r"C:\Program Files\GTK3-Runtime Win64\bin"
# since your Python is definitely 3.8 an your OS is Windows, no need to try...except
os.add_dll_directory(GTK_PATH)

from weasyprint import HTML
HTML('http://weasyprint.org/').write_pdf('weasyprint-website.pdf')

[Tontyna]

os.environ['PATH'] is a string and (initially) contains the %PATH% environment variable. It doesn't check whether the directories in that string exist. E.g. if you SET PATH=foo;bar it will print out foo and bar.

[FaxMeBeer]

Thank you so much Tontyna, and that did work from my Python prompt called from my environment, but not from the windows command prompt within my environment. I am probably just way over my head on this one, and will look to find another tool until I learn more and understand more about what's going on. I appreciate all of your help! Thank you!

[Tontyna]

@FaxMeBeer If the commands work in your Python prompt they'll work in your Windows prompt when you type them into a file, called e.g. cairotest.py and then execute that script file with:

python cairotest.py

in the Windows prompt.

[FaxMeBeer]

Yes, I wrote the file and called it in the prompt, and that did generate the pdf within the directory that I made for the script from the command line!

But, I don't know how to turn that in to a solution that will tell Django where to look, and I'm honestly getting embarrassed at bothering you so much.

[Tontyna]

You don't bother me at all -- it's bothering me that with Python 3.8 it seems no longer possible to use the simple %PATH% method to locate/load the GTK DLLs. There must be a reason for doing that but I didn't yet grasp it. I have vague ideas what happens instead -- but I have no Windows system to test that.

You wrote that outside of your Django project's virtual environment, everything works fine. Where was the GTK3-folder? How do you activate your environment? Are you using PyCharm?

[liZe]

@Tontyna Thanks a lot for the answers :heart:.

it's bothering me that with Python 3.8 it seems no longer possible to use the simple %PATH% method to locate/load the GTK DLLs. There must be a reason for doing that but I didn't yet grasp it.

I think that we don’t use ctypes to find the library. cffi used to call ctypes.util.find_library(), but it doesn’t anymore. We’re only using ctypes.util.find_library() in cairocffi (not in WeasyPrint for Pango, Fontconfig…), but we also call ffi.dlopen() with the filename and it should work as well.

As far as I can tell, ffi.dlopen calls LoadLibraryA on Windows, and LoadLibraryA searches libraries in the PATH.

You wrote that outside of your Django project's virtual environment, everything works fine.

I think that the activate script on Windows changes the PATH environment variable (at least it does on Linux). Could it be the cause of the problem?

[FaxMeBeer]

@Tontyna If you're not bothered, neither am I :)

I am using Visual Studio Code. I can run the script from within my environment, like this:

(collect_it) C:\Users\drcar\collect_it\collect_it>python cairotest/cairo.py

That works fine, it creates the expected PDF under the cairotest subfolder in my project.

but, if I just call WeasyPrint from the command line (within my virtual environment - python -m weasyprint http://weasyprint.org weasyprint.pdf) it still errors out. So, there must be another step that I don't know how to do.

[Tontyna]

As far as I can tell, ffi.dlopen calls LoadLibraryA on Windows, and LoadLibraryA searches libraries in the PATH.

@liZe Of course you are right -- it's ctypes, not ffi that switched to LoadLibraryEx() with parameters to secure the search.

But why for heaven's sake do we get cannot load libary '/full/correct/path/to/gtk/libcairo-2.dll' when this full path (retrieved by ctypes.util.find_library) is the first in the PATH? The answer is probably: Visual Studio already loaded one/some of the helper DLLs that libcairo-2.dll requires and the versions/interface/function definitions don't match.

@FaxMeBeer: AFAIK in Visual Studio you can control the environment variables in the project settings.

[Tontyna]

@FaxMeBeer if you want to run WeasyPrint from the command prompt, outside of Visual Studio, without the need of an extra Python script, without the need of repeatedly typing SET PATH, you should adjust the PATH in your Windows 10 environment variables.