search

getkeops / keops

KErnel OPerationS, on CPUs and GPUs, with autodiff and without memory overflows
https://www.kernel-operations.io
MIT License
1.03k stars 65 forks source link

Python version mismatch when using jupyter despite it being the same version #255

Open kaselby opened 2 years ago

kaselby commented 2 years ago

Hello,

I am trying to use a package that depends on pykeops and have been having immense difficulties getting keops to work without errors on my environment on the gpu server that I run my code on. After a great deal of effort I have now got it to succesfully find cuda and am able to import pykeops without an immediate crash, but I am still unable to get the code I need working in my jupyter notebook.

When I run the following code:

import pykeops
pykeops.test_torch_bindings()

it works fine in the regular python shell of my virtual environment from the terminal, but when I try to run the same thing in a jupyter notebook session that is using the kernel from that same virtual environment it gives me the following error:

ImportError: Python version mismatch: module was compiled for Python 3.6, but the interpreter version is incompatible: 3.7.10 | packaged by conda-forge | (default, Feb 19 2021, 16:07:37) 
[GCC 9.3.0].

When I run sys.version in either the jupyter notebook session or the terminal python session it says 

'3.7.10 | packaged by conda-forge | (default, Feb 19 2021, 16:07:37) \n[GCC 9.3.0]'

which python3 and which pip also confirm that it is finding the python version linked to the correct virtual environment.

Any idea how to fix this?

jeanfeydy commented 1 year ago

Hi @krylea,

Thanks for your interest in the library, and apologies for the delayed answer! Without access to your precise configuration, it is hard to know what is going wrong here. But I would suggest the following strategies:

  1. Try to remove all the cached KeOps binaries: they may have been compiled on your system with Python 3.6 when you tested KeOps in the terminal, and are now faulty when you try to use them with Python 3.7 from your notebook. You should be able to do this with import pykeops; pykeops.clean_pykeops() or, if this fails, manually by deleting a folder called ~/.cache/pykeops2.1_.../ in your home folder.

  2. Alternatively, you may prefer to use a standard environment via a Docker or Singularity container. This may be the simplest option on a GPU server that you do not control entirely: unfortunately, sysadmins in different institutions tend to provide concurrent versions of CUDA or Python in non-standard ways, which causes the issues that you have run into. We provide a standard PyTorch+KeOps container on DockerHub and explain how to use it with Docker and Singularity in the documentation.

What do you think? Best regards, Jean