search

getkeops / keops

KErnel OPerationS, on CPUs and GPUs, with autodiff and without memory overflows
https://www.kernel-operations.io
MIT License
1.03k stars 65 forks source link

EOFError while loading pykeops 2.0 #235

Closed gabrielfougeron closed 2 years ago

gabrielfougeron commented 2 years ago

I get the following weird error when I load pykeops running under wsl2.

---------------------------------------------------------------------------
EOFError                                  Traceback (most recent call last)
/mnt/c/Travail/Python/Optimal transport/test_bar/POC.ipynb Cell 2' in <cell line: 2>()
      [1](vscode-notebook-cell://wsl%2Bubuntu-20.04/mnt/c/Travail/Python/Optimal%20transport/test_bar/POC.ipynb#ch0000001vscode-remote?line=0) get_ipython().run_line_magic('matplotlib', 'inline')
----> [2](vscode-notebook-cell://wsl%2Bubuntu-20.04/mnt/c/Travail/Python/Optimal%20transport/test_bar/POC.ipynb#ch0000001vscode-remote?line=1) from POC import main as POC

File /mnt/c/Travail/Python/Optimal transport/test_bar/POC.py:11, in <module>
      [9](file:///mnt/c/Travail/Python/Optimal%20transport/test_bar/POC.py?line=8) import torch
     [10](file:///mnt/c/Travail/Python/Optimal%20transport/test_bar/POC.py?line=9) from geomloss import SamplesLoss
---> [11](file:///mnt/c/Travail/Python/Optimal%20transport/test_bar/POC.py?line=10) import pykeops
     [12](file:///mnt/c/Travail/Python/Optimal%20transport/test_bar/POC.py?line=11) import chaospy
     [14](file:///mnt/c/Travail/Python/Optimal%20transport/test_bar/POC.py?line=13) from PIL import Image

File ~/anaconda3/envs/ot-geomloss/lib/python3.10/site-packages/pykeops/__init__.py:75, in <module>
     [71](file:///home/gfowsl/anaconda3/envs/ot-geomloss/lib/python3.10/site-packages/pykeops/__init__.py?line=70)     return keops_get_build_folder()
     [74](file:///home/gfowsl/anaconda3/envs/ot-geomloss/lib/python3.10/site-packages/pykeops/__init__.py?line=73) if pykeopsconfig.numpy_found:
---> [75](file:///home/gfowsl/anaconda3/envs/ot-geomloss/lib/python3.10/site-packages/pykeops/__init__.py?line=74)     from .numpy.test_install import test_numpy_bindings
     [77](file:///home/gfowsl/anaconda3/envs/ot-geomloss/lib/python3.10/site-packages/pykeops/__init__.py?line=76) if pykeopsconfig.torch_found:
     [78](file:///home/gfowsl/anaconda3/envs/ot-geomloss/lib/python3.10/site-packages/pykeops/__init__.py?line=77)     from .torch.test_install import test_torch_bindings

File ~/anaconda3/envs/ot-geomloss/lib/python3.[1](file:///home/gfowsl/anaconda3/envs/ot-geomloss/lib/python3.10/site-packages/pykeops/numpy/__init__.py?line=0)0/site-packages/pykeops/numpy/__init__.py:6, in <module>
      1 ##########################################################
      [2](file:///home/gfowsl/anaconda3/envs/ot-geomloss/lib/python3.10/site-packages/pykeops/numpy/__init__.py?line=1) # Import pyKeOps routines
      [5](file:///home/gfowsl/anaconda3/envs/ot-geomloss/lib/python3.10/site-packages/pykeops/numpy/__init__.py?line=4) from .generic.generic_red import Genred
----> [6](file:///home/gfowsl/anaconda3/envs/ot-geomloss/lib/python3.10/site-packages/pykeops/numpy/__init__.py?line=5) from .operations import KernelSolve
      [7](file:///home/gfowsl/anaconda3/envs/ot-geomloss/lib/python3.10/site-packages/pykeops/numpy/__init__.py?line=6) from .generic.generic_ops import (
      [8](file:///home/gfowsl/anaconda3/envs/ot-geomloss/lib/python3.10/site-packages/pykeops/numpy/__init__.py?line=7)     generic_sum,
      [9](file:///home/gfowsl/anaconda3/envs/ot-geomloss/lib/python3.10/site-packages/pykeops/numpy/__init__.py?line=8)     generic_logsumexp,
     [10](file:///home/gfowsl/anaconda3/envs/ot-geomloss/lib/python3.10/site-packages/pykeops/numpy/__init__.py?line=9)     generic_argmin,
     [11](file:///home/gfowsl/anaconda3/envs/ot-geomloss/lib/python3.10/site-packages/pykeops/numpy/__init__.py?line=10)     generic_argkmin,
     [12](file:///home/gfowsl/anaconda3/envs/ot-geomloss/lib/python3.10/site-packages/pykeops/numpy/__init__.py?line=11) )
     [13](file:///home/gfowsl/anaconda3/envs/ot-geomloss/lib/python3.10/site-packages/pykeops/numpy/__init__.py?line=12) from .lazytensor.LazyTensor import LazyTensor, ComplexLazyTensor, Vi, Vj, Pm

File ~/anaconda3/envs/ot-geomloss/lib/python3.[1](file:///home/gfowsl/anaconda3/envs/ot-geomloss/lib/python3.10/site-packages/pykeops/numpy/operations.py?line=0)0/site-packages/pykeops/numpy/operations.py:4, in <module>
      1 import numpy as np
      [3](file:///home/gfowsl/anaconda3/envs/ot-geomloss/lib/python3.10/site-packages/pykeops/numpy/operations.py?line=2) from pykeops.common.get_options import get_tag_backend
----> [4](file:///home/gfowsl/anaconda3/envs/ot-geomloss/lib/python3.10/site-packages/pykeops/numpy/operations.py?line=3) from pykeops.common.keops_io import keops_binder
      [5](file:///home/gfowsl/anaconda3/envs/ot-geomloss/lib/python3.10/site-packages/pykeops/numpy/operations.py?line=4) from pykeops.common.operations import ConjugateGradientSolver
      [6](file:///home/gfowsl/anaconda3/envs/ot-geomloss/lib/python3.10/site-packages/pykeops/numpy/operations.py?line=5) from pykeops.common.parse_type import get_sizes, complete_aliases, get_optional_flags

File ~/anaconda3/envs/ot-geomloss/lib/python3.[1](file:///home/gfowsl/anaconda3/envs/ot-geomloss/lib/python3.10/site-packages/pykeops/common/keops_io/__init__.py?line=0)0/site-packages/pykeops/common/keops_io/__init__.py:4, in <module>
      1 import keopscore.config
      [3](file:///home/gfowsl/anaconda3/envs/ot-geomloss/lib/python3.10/site-packages/pykeops/common/keops_io/__init__.py?line=2) if keopscore.config.config.use_cuda:
----> [4](file:///home/gfowsl/anaconda3/envs/ot-geomloss/lib/python3.10/site-packages/pykeops/common/keops_io/__init__.py?line=3)     from . import LoadKeOps_nvrtc, LoadKeOps_cpp
      [6](file:///home/gfowsl/anaconda3/envs/ot-geomloss/lib/python3.10/site-packages/pykeops/common/keops_io/__init__.py?line=5)     keops_binder = {
      [7](file:///home/gfowsl/anaconda3/envs/ot-geomloss/lib/python3.10/site-packages/pykeops/common/keops_io/__init__.py?line=6)         "nvrtc": LoadKeOps_nvrtc.LoadKeOps_nvrtc,
      [8](file:///home/gfowsl/anaconda3/envs/ot-geomloss/lib/python3.10/site-packages/pykeops/common/keops_io/__init__.py?line=7)         "cpp": LoadKeOps_cpp.LoadKeOps_cpp,
      [9](file:///home/gfowsl/anaconda3/envs/ot-geomloss/lib/python3.10/site-packages/pykeops/common/keops_io/__init__.py?line=8)     }
     [10](file:///home/gfowsl/anaconda3/envs/ot-geomloss/lib/python3.10/site-packages/pykeops/common/keops_io/__init__.py?line=9) else:

File ~/anaconda3/envs/ot-geomloss/lib/python3.10/site-packages/pykeops/common/keops_io/LoadKeOps_nvrtc.py:86, in <module>
     [82](file:///home/gfowsl/anaconda3/envs/ot-geomloss/lib/python3.10/site-packages/pykeops/common/keops_io/LoadKeOps_nvrtc.py?line=81)     KeOps_OS_Run(compile_command)
     [83](file:///home/gfowsl/anaconda3/envs/ot-geomloss/lib/python3.10/site-packages/pykeops/common/keops_io/LoadKeOps_nvrtc.py?line=82)     pyKeOps_Message("OK", use_tag=False, flush=True)
---> [86](file:///home/gfowsl/anaconda3/envs/ot-geomloss/lib/python3.10/site-packages/pykeops/common/keops_io/LoadKeOps_nvrtc.py?line=85) LoadKeOps_nvrtc = Cache_partial(
     [87](file:///home/gfowsl/anaconda3/envs/ot-geomloss/lib/python3.10/site-packages/pykeops/common/keops_io/LoadKeOps_nvrtc.py?line=86)     LoadKeOps_nvrtc_class,
     [88](file:///home/gfowsl/anaconda3/envs/ot-geomloss/lib/python3.10/site-packages/pykeops/common/keops_io/LoadKeOps_nvrtc.py?line=87)     use_cache_file=True,
     [89](file:///home/gfowsl/anaconda3/envs/ot-geomloss/lib/python3.10/site-packages/pykeops/common/keops_io/LoadKeOps_nvrtc.py?line=88)     save_folder=get_build_folder(),
     [90](file:///home/gfowsl/anaconda3/envs/ot-geomloss/lib/python3.10/site-packages/pykeops/common/keops_io/LoadKeOps_nvrtc.py?line=89) )

File ~/anaconda3/envs/ot-geomloss/lib/python3.10/site-packages/keopscore/utils/Cache.py:52, in Cache_partial.__init__(self, cls, use_cache_file, save_folder)
     [50](file:///home/gfowsl/anaconda3/envs/ot-geomloss/lib/python3.10/site-packages/keopscore/utils/Cache.py?line=49) if os.path.isfile(self.cache_file):
     [51](file:///home/gfowsl/anaconda3/envs/ot-geomloss/lib/python3.10/site-packages/keopscore/utils/Cache.py?line=50)     f = open(self.cache_file, "rb")
---> [52](file:///home/gfowsl/anaconda3/envs/ot-geomloss/lib/python3.10/site-packages/keopscore/utils/Cache.py?line=51)     self.library_params = pickle.load(f)
     [53](file:///home/gfowsl/anaconda3/envs/ot-geomloss/lib/python3.10/site-packages/keopscore/utils/Cache.py?line=52)     f.close()
     [54](file:///home/gfowsl/anaconda3/envs/ot-geomloss/lib/python3.10/site-packages/keopscore/utils/Cache.py?line=53) else:

EOFError: Ran out of input
jeanfeydy commented 2 years ago

Hi @gabrielfougeron,

Thanks for your report: we don't use WSL, so some bugs may have slipped in after the v2.0 release.

Just to be sure, are you using KeOps v2.0? Did you install it with pip?

Moreover, I am a little bit surprised to see that from geomloss import SamplesLoss worked fine, but import pykeops crashed. Could you try a minimal script with:

import pykeops
pykeops.test_numpy_bindings()

and show us the error message?

Best regards, Jean

gabrielfougeron commented 2 years ago

Dear Jean,

Thank you very much for your response. For reasons unrelated to this issue, I've had to re-install wsl completely.

The issue might or might not still be there once the install is complete. I'll keep you updated.

for what it's worth, I'm using pykeops (and geomloss) on wsl with another computer, and it works fine there.

Thanks again for your time, and kudos for your amazing work

gabrielfougeron commented 2 years ago

Update :

Indeed, after a clean install of the WSL and everything else, the error disappeared. I really don't know what caused it, but it's very possible it was not related to pykeops at all.

Cheers,

jeanfeydy commented 2 years ago

Hi @gabrielfougeron,

Great, and very happy to know that KeOps can be used without problems on Windows :-)

I'm closing the issue then, feel free to re-open if needed. Best, Jean

mbanani commented 2 years ago

Not sure if it's related. I faced a similar error and it was resolved by deleting the cache folder.

Nucleus2014 commented 1 year ago

I faced the same problem with pykeops 2.1 and it was solved by following @mbanani advice. I just deleted the whole folder of keops under ~/.cache folder by using rm -r ~/.cache/keops2.1.