getkeops / keops

KErnel OPerationS, on CPUs and GPUs, with autodiff and without memory overflows
https://www.kernel-operations.io
MIT License
1.04k stars 64 forks source link

PyKeOps NVRTC binder import fails when using another python interpreter #251

Closed keckj closed 2 years ago

keckj commented 2 years ago

When PyKeOps is run on a different python interpreter version than the one that is bound to python3, the execution will fail due to wrong compiler flags passed during nvrtc binder build step. The failure happens later on first pykeops_nvrtc import.

Offending include dirs are set in pykeops/pykeops/config.py where python3 is hardcoded python_includes = "$(python3 -m pybind11 --includes)" and these flags are later used in pykeops/pykeops/common/keops_io/LoadKeOps_nvrtc.py to build the binder.

Difference between flags for two python versions:

~$ python3 -m pybind11 --includes
-I/usr/include/python3.8 -I/home/keckj/.local/lib/python3.8/site-packages/pybind11/include

~$ python3.9 -m pybind11 --includes
-I/usr/include/python3.9 -I/home/keckj/.local/lib/python3.9/site-packages/pybind11/include

Actual error:


[KeOps] Compiling cuda jit compiler engine ... OK
[pyKeOps] Compiling nvrtc binder for python ... OK
[KeOps] Generating code for formula Max_SumShiftExpWeight_Reduction(Concat(Var(2,1,1)-Var(3,1,2)*Sqrt((Var(0,3,0)-Var(1,3,1))|(Var(0,3,0)-Var(1,3,1))),1),0) ... OK
Traceback (most recent call last):
  ...
  File ".../lib/python3.9/site-packages/pykeops/common/keops_io/LoadKeOps_nvrtc.py", line 20, in init_phase2
    pykeops_nvrtc = importlib.import_module("pykeops_nvrtc")
  File "/usr/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 666, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 565, in module_from_spec
  File "<frozen importlib._bootstrap_external>", line 1173, in create_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
ImportError: Python version mismatch: module was compiled for Python 3.8, but the interpreter version is incompatible: 3.9.5 (default, Nov 23 2021, 15:27:38) 
[GCC 9.3.0].
joanglaunes commented 2 years ago

Thank you very much @keckj for fixing this, I have merged your pull request now and everything works.