search

docker-library / python

Docker Official Image packaging for Python
https://www.python.org/
MIT License
2.53k stars 1.06k forks source link

unable to run vtk within windowsservercore-ltsc2022⁠ #956

Closed mutricyl closed 1 week ago

mutricyl commented 2 weeks ago

I try to run a python project using vtk within a python:windowsservercore-ltsc2022⁠ docker image (actually these are gitlab CI/CD tasks for this project). Unfortunately I am not able to import vtk. It always fail with the following error:

>>> import vtk
ImportError: DLL load failed while importing vtkWebCore: The specified module could not be found.

It seams that python can not load vtk specific dll files (located in site-packages\vtk.lib). Of course doing the same on a clean install outside of docker is working perfectly so I wonder if :

LaurentGoderre commented 2 weeks ago

How are you getting vtk in the image?

mutricyl commented 2 weeks ago

in the gitlab process vtk is installed with pip before running the scripts. The python docker image is used as such by gitlab runner.

This is latest vtk (3.9.1) and python 3.12 by the way.

LaurentGoderre commented 2 weeks ago

And is it installed with pip in the image or mounted in?

mutricyl commented 2 weeks ago

Installed in the image at run time.

tianon commented 2 weeks ago

I was able to reproduce, but unfortunately don't have much insight beyond that: :disappointed:

$ docker run -it --rm --pull=always python:3.12-windowsservercore powershell
3.12-windowsservercore: Pulling from library/python
Digest: sha256:f561ab0c4ac0b3f8d97ca1880e8c86088a3049f150d0d43f0168b99a7ff89683
Status: Image is up to date for python:3.12-windowsservercore

Windows PowerShell
Copyright (C) Microsoft Corporation. All rights reserved.

Install the latest PowerShell for new features and improvements! https://aka.ms/PSWindows

PS C:\> pip install vtk
Collecting vtk
  Downloading vtk-9.3.1-cp312-cp312-win_amd64.whl.metadata (5.3 kB)
Collecting matplotlib>=2.0.0 (from vtk)
  Downloading matplotlib-3.9.2-cp312-cp312-win_amd64.whl.metadata (11 kB)
Collecting contourpy>=1.0.1 (from matplotlib>=2.0.0->vtk)
  Downloading contourpy-1.3.0-cp312-cp312-win_amd64.whl.metadata (5.4 kB)
Collecting cycler>=0.10 (from matplotlib>=2.0.0->vtk)
  Downloading cycler-0.12.1-py3-none-any.whl.metadata (3.8 kB)
Collecting fonttools>=4.22.0 (from matplotlib>=2.0.0->vtk)
  Downloading fonttools-4.53.1-cp312-cp312-win_amd64.whl.metadata (165 kB)
Collecting kiwisolver>=1.3.1 (from matplotlib>=2.0.0->vtk)
  Downloading kiwisolver-1.4.6-cp312-cp312-win_amd64.whl.metadata (6.4 kB)
Collecting numpy>=1.23 (from matplotlib>=2.0.0->vtk)
  Downloading numpy-2.1.1-cp312-cp312-win_amd64.whl.metadata (59 kB)
Collecting packaging>=20.0 (from matplotlib>=2.0.0->vtk)
  Downloading packaging-24.1-py3-none-any.whl.metadata (3.2 kB)
Collecting pillow>=8 (from matplotlib>=2.0.0->vtk)
  Downloading pillow-10.4.0-cp312-cp312-win_amd64.whl.metadata (9.3 kB)
Collecting pyparsing>=2.3.1 (from matplotlib>=2.0.0->vtk)
  Downloading pyparsing-3.1.4-py3-none-any.whl.metadata (5.1 kB)
Collecting python-dateutil>=2.7 (from matplotlib>=2.0.0->vtk)
  Downloading python_dateutil-2.9.0.post0-py2.py3-none-any.whl.metadata (8.4 kB)
Collecting six>=1.5 (from python-dateutil>=2.7->matplotlib>=2.0.0->vtk)
  Downloading six-1.16.0-py2.py3-none-any.whl.metadata (1.8 kB)
Downloading vtk-9.3.1-cp312-cp312-win_amd64.whl (52.5 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 52.5/52.5 MB 9.0 MB/s eta 0:00:00   
Downloading matplotlib-3.9.2-cp312-cp312-win_amd64.whl (7.8 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7.8/7.8 MB 81.4 MB/s eta 0:00:00    
Downloading contourpy-1.3.0-cp312-cp312-win_amd64.whl (218 kB)
Downloading cycler-0.12.1-py3-none-any.whl (8.3 kB)
Downloading fonttools-4.53.1-cp312-cp312-win_amd64.whl (2.2 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.2/2.2 MB 62.3 MB/s eta 0:00:00    
Downloading kiwisolver-1.4.6-cp312-cp312-win_amd64.whl (55 kB)
Downloading numpy-2.1.1-cp312-cp312-win_amd64.whl (12.6 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 12.6/12.6 MB 41.4 MB/s eta 0:00:00  
Downloading packaging-24.1-py3-none-any.whl (53 kB)
Downloading pillow-10.4.0-cp312-cp312-win_amd64.whl (2.6 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.6/2.6 MB 74.0 MB/s eta 0:00:00    
Downloading pyparsing-3.1.4-py3-none-any.whl (104 kB)
Downloading python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB)
Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
Installing collected packages: six, pyparsing, pillow, packaging, numpy, kiwisolver, fonttools, cycler, python-dateutil, contourpy, matplotlib, vtk
Successfully installed contourpy-1.3.0 cycler-0.12.1 fonttools-4.53.1 kiwisolver-1.4.6 matplotlib-3.9.2 numpy-2.1.1 packaging-24.1 pillow-10.4.0 pyparsing-3.1.4 python-dateutil-2.9.0.post0 six-1.16.0 vtk-9.3.1
PS C:\> python
Python 3.12.5 (tags/v3.12.5:ff3bc82, Aug  6 2024, 20:45:27) [MSC v.1940 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import vtk
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Python\Lib\site-packages\vtk.py", line 5, in <module>
    from vtkmodules.vtkWebCore import *
ImportError: DLL load failed while importing vtkWebCore: The specified module could not be found.

This seems like something that Python / pip install should be handling for us, but I do have to admit I haven't done much with Python on Windows (and certainly not modules that require extra DLLs :sweat_smile:).

tianon commented 2 weeks ago

I wonder if it's perhaps actually complaining about https://docs.vtk.org/en/latest/getting_started/index.html#:~:text=Minimum%20OpenGL%20version%20is%203.2%20but%20a%20higher%20versions%20may%20be%20required%20for%20more%20advanced%20features. ?

Minimum OpenGL version is 3.2 but a higher versions may be required for more advanced features.

(I don't know -- that page is pretty light on Windows-specific requirements :sweat_smile:)

LaurentGoderre commented 2 weeks ago

This seems to be happening because some librar(y)(ies) that WebCore are linked to are missing. It seems to be depending on the C++ redistributable but that didn't fix it on its own

mutricyl commented 2 weeks ago

I'll try to explore the openGL option. Pyvista doc for headless setups lead me to this script: https://github.com/pyvista/setup-headless-display-action/blob/main/windows/install_opengl.ps1

LaurentGoderre commented 1 week ago

I;m wondering if it is possible to do this because the Microsoft docs say that only DirectX is supported.

https://learn.microsoft.com/en-us/virtualization/windowscontainers/deploy-containers/gpu-acceleration#run-a-container-with-gpu-acceleration

LaurentGoderre commented 1 week ago

I tracked it down to a missing opengl library in the image (opengl32.dll)

LaurentGoderre commented 1 week ago

You could try installing Mesa3D as mentioned in the opengl docs

https://www.khronos.org/opengl/wiki/Platform_specifics:_Windows#Installing_Mesa3D_on_Windows

mutricyl commented 1 week ago

pyvista script does indeed download some Mesa related opengl32.dll file. And this is sufficient, I can now import vtk without any error. Thanks so much @LaurentGoderre and @tianon for the help on this issue. You made my day !

LaurentGoderre commented 1 week ago

Can you share what exactly you downloaded so if someone else can follow our steps?

mutricyl commented 1 week ago

For the record pyvista documentation also applies for vtk itself. It can be directly used for people not in a position to use a custom docker image.

I personally build a docker image downloading directly the proper opengl32.dll file from pyvista:

FROM python:3.12.3-windowsservercore-ltsc2022

RUN [Net.ServicePointManager]::SecurityProtocol = 'Tls, Tls11, Tls12';
RUN Invoke-WebRequest "https://github.com/vispy/demo-data/raw/main/mesa/opengl32_mingw_64.dll" -OutFile "C:\Windows\system32\opengl32.dll";

CMD ["python"]