EOFError : Ran out of input when importing pykeops on ubuntu

[alexandre-blanc]

Hi !

I want to use pykeops on ubuntu 18.04, with python 3.8. I get the same exception as in issue #235 https://github.com/getkeops/keops/issues/235.

Below is the output of the installation process in a fresh new virtual environment :

➜  workspace which python3    
/usr/bin/python3
➜  workspace python3 --version
Python 3.6.9
➜  workspace python3 -m virtualenv --python=/usr/bin/python3.8 test_pykeops_venv 
created virtual environment CPython3.8.13.final.0-64 in 229ms
  creator CPython3Posix(dest=/home/alexandre/workspace/test_pykeops_venv, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/alexandre/.local/share/virtualenv)
    added seed packages: pip==22.0.4, setuptools==62.1.0, wheel==0.37.1
  activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator
➜  workspace source test_pykeops_venv/bin/activate 
(test_pykeops_venv) ➜  workspace pip --version 
pip 22.0.4 from /home/alexandre/workspace/test_pykeops_venv/lib/python3.8/site-packages/pip (python 3.8)
(test_pykeops_venv) ➜  workspace python --version
Python 3.8.13
(test_pykeops_venv) ➜  workspace pip install pykeops
Collecting pykeops
  Using cached pykeops-2.0-py3-none-any.whl
Collecting numpy
  Using cached numpy-1.22.3-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (16.8 MB)
Collecting pybind11
  Using cached pybind11-2.9.2-py2.py3-none-any.whl (213 kB)
Collecting keopscore==2.0
  Using cached keopscore-2.0-py3-none-any.whl
Installing collected packages: pybind11, numpy, keopscore, pykeops
Successfully installed keopscore-2.0 numpy-1.22.3 pybind11-2.9.2 pykeops-2.0
(test_pykeops_venv) ➜  workspace python
Python 3.8.13 (default, Apr 19 2022, 00:53:22) 
[GCC 7.5.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pykeops
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/alexandre/workspace/test_pykeops_venv/lib/python3.8/site-packages/pykeops/__init__.py", line 75, in <module>
    from .numpy.test_install import test_numpy_bindings
  File "/home/alexandre/workspace/test_pykeops_venv/lib/python3.8/site-packages/pykeops/numpy/__init__.py", line 6, in <module>
    from .operations import KernelSolve
  File "/home/alexandre/workspace/test_pykeops_venv/lib/python3.8/site-packages/pykeops/numpy/operations.py", line 4, in <module>
    from pykeops.common.keops_io import keops_binder
  File "/home/alexandre/workspace/test_pykeops_venv/lib/python3.8/site-packages/pykeops/common/keops_io/__init__.py", line 4, in <module>
    from . import LoadKeOps_nvrtc, LoadKeOps_cpp
  File "/home/alexandre/workspace/test_pykeops_venv/lib/python3.8/site-packages/pykeops/common/keops_io/LoadKeOps_nvrtc.py", line 8, in <module>
    from pykeops.common.keops_io.LoadKeOps import LoadKeOps
  File "/home/alexandre/workspace/test_pykeops_venv/lib/python3.8/site-packages/pykeops/common/keops_io/LoadKeOps.py", line 6, in <module>
    from keopscore.get_keops_dll import get_keops_dll
  File "/home/alexandre/workspace/test_pykeops_venv/lib/python3.8/site-packages/keopscore/get_keops_dll.py", line 150, in <module>
    get_keops_dll = Cache(
  File "/home/alexandre/workspace/test_pykeops_venv/lib/python3.8/site-packages/keopscore/utils/Cache.py", line 18, in __init__
    self.library = pickle.load(f)
EOFError: Ran out of input
>>> 

When downgrading to pykeops 1.5 however, the import doesn't throw any exceptions.

[jeanfeydy]

Hi @alexandre-blanc,

Thanks for your interest in this library.

To be clear: issue #235 is about using KeOps with WSL2, and got closed since the issue got resolved after a reinstall of WSL. (I can confirm that KeOps 2.0 should work just fine on a fresh Windows+WSL install.) Are you using the same kind of hybrid Windows+Ubuntu system, or are you working on a straight Linux configuration?

With respect to the pickle issue: do @joanglaunes or @bcharlier have any idea? I have never seen this bug, beyond #235.

Best regards, Jean

[alexandre-blanc]

Hi @jeanfeydy, this error pops up on a ubuntu workstation, with ubuntu being the sole OS installed. There is no link to WSL. I only referenced #235 because the exception is identical and I thought it may help.

Best, Alexandre

[xlliu2017]

Hi @alexandre-blanc , @jeanfeydy
I have similar issue when import pykeops on ubuntu cluster. I only have this problem when import pykeops on multi-gpus nodes.

`----> 1 import pykeops

File ~/miniconda3/envs/pyg/lib/python3.9/site-packages/pykeops/init.py:75, in 71 return keops_get_build_folder() 74 if pykeopsconfig.numpy_found: ---> 75 from .numpy.test_install import test_numpy_bindings 77 if pykeopsconfig.torch_found: 78 from .torch.test_install import test_torch_bindings

File ~/miniconda3/envs/pyg/lib/python3.9/site-packages/pykeops/numpy/init.py:6, in 1 ########################################################## 2 # Import pyKeOps routines 5 from .generic.generic_red import Genred ----> 6 from .operations import KernelSolve 7 from .generic.generic_ops import ( 8 generic_sum, 9 generic_logsumexp, 10 generic_argmin, 11 generic_argkmin, 12 ) 13 from .lazytensor.LazyTensor import LazyTensor, ComplexLazyTensor, Vi, Vj, Pm

File ~/miniconda3/envs/pyg/lib/python3.9/site-packages/pykeops/numpy/operations.py:4, in 1 import numpy as np 3 from pykeops.common.get_options import get_tag_backend ----> 4 from pykeops.common.keops_io import keops_binder 5 from pykeops.common.operations import ConjugateGradientSolver 6 from pykeops.common.parse_type import get_sizes, complete_aliases, get_optional_flags

File ~/miniconda3/envs/pyg/lib/python3.9/site-packages/pykeops/common/keops_io/init.py:4, in 1 import keopscore.config 3 if keopscore.config.config.use_cuda: ----> 4 from . import LoadKeOps_nvrtc, LoadKeOps_cpp 6 keops_binder = { 7 "nvrtc": LoadKeOps_nvrtc.LoadKeOps_nvrtc, 8 "cpp": LoadKeOps_cpp.LoadKeOps_cpp, 9 } 10 else:

File ~/miniconda3/envs/pyg/lib/python3.9/site-packages/pykeops/common/keops_io/LoadKeOps_nvrtc.py:8, in 6 from keopscore.binders.nvrtc.Gpu_link_compile import Gpu_link_compile 7 from keopscore.utils.Cache import Cache_partial ----> 8 from pykeops.common.keops_io.LoadKeOps import LoadKeOps 9 from pykeops.common.utils import pyKeOps_Message 10 from keopscore.utils.misc_utils import KeOps_OS_Run

File ~/miniconda3/envs/pyg/lib/python3.9/site-packages/pykeops/common/keops_io/LoadKeOps.py:6, in 2 from functools import reduce 4 import numpy as np ----> 6 from keopscore.get_keops_dll import get_keops_dll 7 from pykeops.common.parse_type import parse_dtype_acc 10 class LoadKeOps:

File ~/miniconda3/envs/pyg/lib/python3.9/site-packages/keopscore/get_keops_dll.py:150, in 126 tag1D2D = 0 if tagZero == 1 else res["tag1D2D"] 128 return ( 129 res["tag"], 130 res["source_file"], (...) 146 res["dimsp"], 147 ) --> 150 get_keops_dll = Cache( 151 get_keops_dll_impl, 152 use_cache_file=True, 153 save_folder=get_build_folder(), 154 ) 157 if name == "main": 158 argv = sys.argv[1:]

File ~/miniconda3/envs/pyg/lib/python3.9/site-packages/keopscore/utils/Cache.py:18, in Cache.init(self, fun, use_cache_file, save_folder) 16 if os.path.isfile(self.cache_file): 17 f = open(self.cache_file, "rb") ---> 18 self.library = pickle.load(f) 19 f.close() 20 import atexit

EOFError: Ran out of input`

[joanglaunes]

Hello @alexandre-blanc , I have tried to fix the issue by just checking the length of the cache file before reading it. I am not sure that it will really fix it, because I cannot reproduce your error. Could you try it (the fix is commited in the main branch) ?

[miderxi]

try: pip uninstall pykeops pip install pykeops==1.5

[daskandalis]

I encountered precisely the same issue. I had a working conda environment with pykeops 2.1, which suddenly started throwing this error. The only thing I can think of is that since last using that environment, I created other environments with pytorch and cuda (probably with pip). Possibly this is the source of a conflict?

pip install pykeops<2.1 fixed the issue in the current environment.

[miderxi]

我已收到邮件，会尽快处理。

[FreyrS]

I ran into this issue when keops was first trying to compile a formula but ran into a Disk out-of-space error. This seems to have left some files in the .cache/ folder corrupted and when I tried to import pykeops later I got the EOFError: Ran out of input. The solution was to remove the .cache folder manually before importing again.

[miderxi]

我已收到邮件，会尽快处理。