search

mitsuba-renderer / enoki

Enoki: structured vectorization and differentiation on modern processor architectures
Other
1.26k stars 94 forks source link

Python ImportError on Windows with enoki.cuda #120

Closed petomka closed 3 years ago

petomka commented 3 years ago

Importing the enoki.cuda module from python results in an ImportError (described below). My setup is the following:

I am on Windows Version 2004 Build 19041.1052. To compile the system, I use the MSVC toolchain (cl Version 19.28.29915 for x64). My directory setup after compiling looks as follows:

├── cmake_install.cmake
├── CMakeFiles/
│   └── (...)
├── core.exp
├── core.lib
├── cuda.exp
├── cuda.lib
├── enoki/
│   ├── __init__.py
│   ├── __pycache__/
│   │   └── __init__.cpython-39.pyc
│   ├── core.cp39-win_amd64.pyd
│   ├── core.cp39-win_amd64.pyd.manifest
│   ├── cuda.cp39-win_amd64.pyd
│   ├── cuda.cp39-win_amd64.pyd.manifest
│   └── enoki-cuda.dll
├── enoki-cuda.dll
├── enoki-cuda.dll.manifest
├── enoki-cuda.exp
├── enoki-cuda.lib
├── enoki.cbp
├── Makefile
└── pybind11/
    └── (...)

(I use enoki inside of another project, that's why there is other stuff too. I get the same results with pure enoki as standalone project.)

I copied the enoki-cuda.dll manually inside the enoki directory to avoid possible loading issues (will become clear in a bit).

My Python version: Python 3.9.6 (tags/v3.9.6:db3ff76, Jun 28 2021, 15:26:21) [MSC v.1929 64 bit (AMD64)] on win32

I run my python3 from the directory as seen above. When I try to import enoki, it works fine, and the help message (>>> help(enoki) is also working. However, importing enoki.cuda yields the following error:

>>> import enoki.cuda
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: DLL load failed while importing cuda: Das angegebene Modul wurde nicht gefunden.

(translation from german: ImportError: DLL load failed: The specified module could not be found.)

To resolve this issue, I checked the cuda.cp39-win_amd64.pyd in every way that was suggested and I could think of. This includes:

My PATH looks like this:

and my CUDA environment variables look like this:

Using the enoki-Library as just a C++-Library works perfectly fine for me, including CUDA features. I have also tried compiling enoki-python on Linux on my machine and it works perfectly there, it just does not want to work with Windows. I would really like to be able to have the application I am developing for my Bachelor's Thesis also working on windows.

Speierers commented 3 years ago

Hi @petomka ,

Could you maybe try following the Mitsuba2 documentation? Could be that the enoki doc is outdated for building on Windows.

https://mitsuba2.readthedocs.io/en/latest/src/getting_started/compiling.html

petomka commented 3 years ago

Hi @Speierers,

I pulled and compiled Mitsuba2 as stated in the documentation. For the variants I kept the two default values and added gpu_autodiff_rgb. I set the build mode to Release and the system compiled fine. Before I tested the system with python, I wanted to make the binary would work. To test this, I downloaded the bunny.zip from this Mitsuba2 doc page. I opened a new terminal, ran setpath.bat in it and let bunny.xml render, which it did just fine (using the command mitsuba -m gpu_autodiff_rgb bunny.xml, because I did not set the default). The resulting image bunny.exr looked exactly as expected, the rendering itself produced no errors (only warnings).

Then, in the same terminal window (which had setpath already executed) I started a new python3 process and tried to import mitsuba, the result of which is the following:

Python 3.9.6 (tags/v3.9.6:db3ff76, Jun 28 2021, 15:26:21) [MSC v.1929 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import mitsuba
Traceback (most recent call last):
  File "C:\mitsuba2\dist\python\mitsuba\__init__.py", line 12, in <module>
    _import('mitsuba.core_ext')
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.1776.0_x64__qbz5n2kfra8p0\lib\importlib\__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 666, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 565, in module_from_spec
  File "<frozen importlib._bootstrap_external>", line 1173, in create_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
ImportError: DLL load failed while importing core_ext: Das angegebene Modul wurde nicht gefunden.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\mitsuba2\dist\python\mitsuba\__init__.py", line 33, in <module>
    raise exc
ImportError: The 'mitsuba' native modules could not be imported.
>>> exit()

This was done using Python from the Microsoft Store. After that, I went back to the documentation and realized that Mitsuba2 was stating testing was done using Miniconda. So I uninstalled the Python from Microsoft and installed the latest Miniconda (which uses Python 3.9.1), repeated the steps and - IT WORKED. Using Miniconda I could import mitsuba and enoki.cuda, also in my project for my Bachelor's Thesis.

I will continue using Miniconda, but I would still like to understand what the actual problem was. Anyone else that runs into similar issues can just try using Miniconda if they are not already using it.

So while not directly solving the issue, thank you anyways for bringing a fresh idea to the table that helped me solve it @Speierers 👍