search

getkeops / keops

KErnel OPerationS, on CPUs and GPUs, with autodiff and without memory overflows
https://www.kernel-operations.io
MIT License
1.04k stars 64 forks source link

PyKeops can't be used even on CPU #201

Open izzatum opened 2 years ago

izzatum commented 2 years ago

Hi! I have read and tried every possible way I found in the issues posted to make this package usable. However, I still can't make this work. I keep receiving the following error:

` pykeops.test_numpy_bindings() [pyKeOps] Initializing build folder for dtype=float64 and lang=numpy in /Users/izzatum/.cache/pykeops-1.5-cpython-38 ... done. [pyKeOps] Compiling libKeOpsnumpy8c9488f264 in /Users/izzatum/.cache/pykeops-1.5-cpython-38: formula: Sum_Reduction(SqNorm2(x - y),1) aliases: x = Vi(0,3); y = Vj(1,3); dtype : float64 ... In file included from :1: In file included from /Users/izzatum/.cache/pykeops-1.5-cpython-38/build-0b4fcbd2ce/KeOps_formula.h:13: In file included from /Users/izzatum/opt/anaconda3/lib/python3.8/site-packages/pykeops/cmake_scripts/script_keops_formula/../../keops/keops_includes.h:46: In file included from /Users/izzatum/opt/anaconda3/lib/python3.8/site-packages/pykeops/cmake_scripts/script_keops_formula/../../keops/core/formulas/constants/IntConst.h:3: In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/sstream:173: In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/ostream:138: In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/ios:215: In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/locale:14: In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/string:506: In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/string_view:175: In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/string:57: In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/algorithm:639: In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/initializer_list:46: In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/cstddef:37: /Users/izzatum/opt/anaconda3/lib/python3.8/site-packages/pykeops/cmake_scripts/script_keops_formula/../../version:1:1: error: expected unqualified-id 1.5 ^ In file included from :1: In file included from /Users/izzatum/.cache/pykeops-1.5-cpython-38/build-0b4fcbd2ce/KeOps_formula.h:13: In file included from /Users/izzatum/opt/anaconda3/lib/python3.8/site-packages/pykeops/cmake_scripts/script_keops_formula/../../keops/keops_includes.h:46: In file included from /Users/izzatum/opt/anaconda3/lib/python3.8/site-packages/pykeops/cmake_scripts/script_keops_formula/../../keops/core/formulas/constants/IntConst.h:3: In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/sstream:173: In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/ostream:138: In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/ios:215: In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/locale:14: In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/string:506: In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/string_view:175: In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/string:57: In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/algorithm:640: In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/type_traits:418: /Users/izzatum/opt/anaconda3/lib/python3.8/site-packages/pykeops/cmake_scripts/script_keops_formula/../../version:1:1: error: expected unqualified-id 1.5 ^ fatal error: too many errors emitted, stopping now [-ferror-limit=] 3 errors generated. make[3]: [CMakeFiles/copy_KeOps_formula.dir/Users/izzatum/opt/anaconda3/lib/python3.8/site-packages/pykeops/keops/core/link_autodiff.cpp.o] Error 1 make[2]: [CMakeFiles/copy_KeOps_formula.dir/all] Error 2 make[1]: [CMakeFiles/KeOps_formula.dir/rule] Error 2 make: [KeOps_formula] Error 2

--------------------- MAKE DEBUG ----------------- Command '['cmake', '--build', '.', '--target', 'KeOps_formula', '--', 'VERBOSE=1']' returned non-zero exit status 2. /Users/izzatum/opt/anaconda3/bin/cmake -S/Users/izzatum/opt/anaconda3/lib/python3.8/site-packages/pykeops/cmake_scripts/script_keops_formula -B/Users/izzatum/.cache/pykeops-1.5-cpython-38/build-0b4fcbd2ce --check-build-system CMakeFiles/Makefile.cmake 0 /Library/Developer/CommandLineTools/usr/bin/make -f CMakeFiles/Makefile2 KeOps_formula /Users/izzatum/opt/anaconda3/bin/cmake -S/Users/izzatum/opt/anaconda3/lib/python3.8/site-packages/pykeops/cmake_scripts/script_keops_formula -B/Users/izzatum/.cache/pykeops-1.5-cpython-38/build-0b4fcbd2ce --check-build-system CMakeFiles/Makefile.cmake 0 /Users/izzatum/opt/anaconda3/bin/cmake -E cmake_progress_start /Users/izzatum/.cache/pykeops-1.5-cpython-38/build-0b4fcbd2ce/CMakeFiles 1 /Library/Developer/CommandLineTools/usr/bin/make -f CMakeFiles/Makefile2 CMakeFiles/KeOps_formula.dir/all /Library/Developer/CommandLineTools/usr/bin/make -f CMakeFiles/copy_KeOps_formula.dir/build.make CMakeFiles/copy_KeOps_formula.dir/depend cd /Users/izzatum/.cache/pykeops-1.5-cpython-38/build-0b4fcbd2ce && /Users/izzatum/opt/anaconda3/bin/cmake -E cmake_depends "Unix Makefiles" /Users/izzatum/opt/anaconda3/lib/python3.8/site-packages/pykeops/cmake_scripts/script_keops_formula /Users/izzatum/opt/anaconda3/lib/python3.8/site-packages/pykeops/cmake_scripts/script_keops_formula /Users/izzatum/.cache/pykeops-1.5-cpython-38/build-0b4fcbd2ce /Users/izzatum/.cache/pykeops-1.5-cpython-38/build-0b4fcbd2ce /Users/izzatum/.cache/pykeops-1.5-cpython-38/build-0b4fcbd2ce/CMakeFiles/copy_KeOps_formula.dir/DependInfo.cmake --color= /Library/Developer/CommandLineTools/usr/bin/make -f CMakeFiles/copy_KeOps_formula.dir/build.make CMakeFiles/copy_KeOps_formula.dir/build [100%] Building CXX object CMakeFiles/copy_KeOps_formula.dir/Users/izzatum/opt/anaconda3/lib/python3.8/site-packages/pykeops/keops/core/link_autodiff.cpp.o /Library/Developer/CommandLineTools/usr/bin/c++ -DC_CONTIGUOUS=1 -DUSE_CUDA=0 -DUSE_DOUBLE=1 -DUSE_HALF=0 -DTYPE=double -I/Users/izzatum/opt/anaconda3/lib/python3.8/site-packages/pykeops/cmake_scripts/script_keops_formula/../.. -I/Users/izzatum/.cache/pykeops-1.5-cpython-38/build-0b4fcbd2ce -I/Users/izzatum/opt/anaconda3/lib/python3.8/site-packages/pykeops/cmake_scripts/script_keops_formula/../../keops -ferror-limit=2 -O3 -DNDEBUG -O3 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk -fPIC -include KeOps_formula.h -std=gnu++14 -MD -MT CMakeFiles/copy_KeOps_formula.dir/Users/izzatum/opt/anaconda3/lib/python3.8/site-packages/pykeops/keops/core/link_autodiff.cpp.o -MF CMakeFiles/copy_KeOps_formula.dir/Users/izzatum/opt/anaconda3/lib/python3.8/site-packages/pykeops/keops/core/link_autodiff.cpp.o.d -o CMakeFiles/copy_KeOps_formula.dir/Users/izzatum/opt/anaconda3/lib/python3.8/site-packages/pykeops/keops/core/link_autodiff.cpp.o -c /Users/izzatum/opt/anaconda3/lib/python3.8/site-packages/pykeops/keops/core/link_autodiff.cpp

[pyKeOps] Compiling pybind11 template libKeOps_template_16fe3306d9 in /Users/izzatum/.cache/pykeops-1.5-cpython-38 ... [pyKeOps] Initializing build folder for dtype=float64 and lang=numpy in /Users/izzatum/.cache/pykeops-1.5-cpython-38 ... -- The CXX compiler identification is AppleClang 13.0.0.13000029 -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Check for working CXX compiler: /Library/Developer/CommandLineTools/usr/bin/c++ - skipped -- Detecting CXX compile features -- Detecting CXX compile features - done -- Looking for a CUDA compiler -- Looking for a CUDA compiler - NOTFOUND -- No CUDA support -- Configuring done -- Generating done -- Build files have been written to: /Users/izzatum/.cache/pykeops-1.5-cpython-38/build-cafbf708bb

done. [pyKeOps] Compiling libKeOpsnumpy8c9488f264 in /Users/izzatum/.cache/pykeops-1.5-cpython-38: formula: Sum_Reduction(SqNorm2(x - y),1) aliases: x = Vi(0,3); y = Vj(1,3); dtype : float64 ... In file included from :1: In file included from /Users/izzatum/.cache/pykeops-1.5-cpython-38/build-cafbf708bb/KeOps_formula.h:13: In file included from /Users/izzatum/opt/anaconda3/lib/python3.8/site-packages/pykeops/cmake_scripts/script_keops_formula/../../keops/keops_includes.h:46: In file included from /Users/izzatum/opt/anaconda3/lib/python3.8/site-packages/pykeops/cmake_scripts/script_keops_formula/../../keops/core/formulas/constants/IntConst.h:3: In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/sstream:173: In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/ostream:138: In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/ios:215: In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/locale:14: In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/string:506: In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/string_view:175: In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/string:57: In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/algorithm:639: In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/initializer_list:46: In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/cstddef:37: /Users/izzatum/opt/anaconda3/lib/python3.8/site-packages/pykeops/cmake_scripts/script_keops_formula/../../version:1:1: error: expected unqualified-id 1.5 ^ In file included from :1: In file included from /Users/izzatum/.cache/pykeops-1.5-cpython-38/build-cafbf708bb/KeOps_formula.h:13: In file included from /Users/izzatum/opt/anaconda3/lib/python3.8/site-packages/pykeops/cmake_scripts/script_keops_formula/../../keops/keops_includes.h:46: In file included from /Users/izzatum/opt/anaconda3/lib/python3.8/site-packages/pykeops/cmake_scripts/script_keops_formula/../../keops/core/formulas/constants/IntConst.h:3: In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/sstream:173: In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/ostream:138: In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/ios:215: In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/locale:14: In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/string:506: In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/string_view:175: In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/string:57: In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/algorithm:640: In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/type_traits:418: /Users/izzatum/opt/anaconda3/lib/python3.8/site-packages/pykeops/cmake_scripts/script_keops_formula/../../version:1:1: error: expected unqualified-id 1.5 ^ fatal error: too many errors emitted, stopping now [-ferror-limit=] 3 errors generated. make[3]: [CMakeFiles/copy_KeOps_formula.dir/Users/izzatum/opt/anaconda3/lib/python3.8/site-packages/pykeops/keops/core/link_autodiff.cpp.o] Error 1 make[2]: [CMakeFiles/copy_KeOps_formula.dir/all] Error 2 make[1]: [CMakeFiles/KeOps_formula.dir/rule] Error 2 make: [KeOps_formula] Error 2

--------------------- MAKE DEBUG ----------------- Command '['cmake', '--build', '.', '--target', 'KeOps_formula', '--', 'VERBOSE=1']' returned non-zero exit status 2. /Users/izzatum/opt/anaconda3/bin/cmake -S/Users/izzatum/opt/anaconda3/lib/python3.8/site-packages/pykeops/cmake_scripts/script_keops_formula -B/Users/izzatum/.cache/pykeops-1.5-cpython-38/build-cafbf708bb --check-build-system CMakeFiles/Makefile.cmake 0 /Library/Developer/CommandLineTools/usr/bin/make -f CMakeFiles/Makefile2 KeOps_formula /Users/izzatum/opt/anaconda3/bin/cmake -S/Users/izzatum/opt/anaconda3/lib/python3.8/site-packages/pykeops/cmake_scripts/script_keops_formula -B/Users/izzatum/.cache/pykeops-1.5-cpython-38/build-cafbf708bb --check-build-system CMakeFiles/Makefile.cmake 0 /Users/izzatum/opt/anaconda3/bin/cmake -E cmake_progress_start /Users/izzatum/.cache/pykeops-1.5-cpython-38/build-cafbf708bb/CMakeFiles 1 /Library/Developer/CommandLineTools/usr/bin/make -f CMakeFiles/Makefile2 CMakeFiles/KeOps_formula.dir/all /Library/Developer/CommandLineTools/usr/bin/make -f CMakeFiles/copy_KeOps_formula.dir/build.make CMakeFiles/copy_KeOps_formula.dir/depend cd /Users/izzatum/.cache/pykeops-1.5-cpython-38/build-cafbf708bb && /Users/izzatum/opt/anaconda3/bin/cmake -E cmake_depends "Unix Makefiles" /Users/izzatum/opt/anaconda3/lib/python3.8/site-packages/pykeops/cmake_scripts/script_keops_formula /Users/izzatum/opt/anaconda3/lib/python3.8/site-packages/pykeops/cmake_scripts/script_keops_formula /Users/izzatum/.cache/pykeops-1.5-cpython-38/build-cafbf708bb /Users/izzatum/.cache/pykeops-1.5-cpython-38/build-cafbf708bb /Users/izzatum/.cache/pykeops-1.5-cpython-38/build-cafbf708bb/CMakeFiles/copy_KeOps_formula.dir/DependInfo.cmake --color= /Library/Developer/CommandLineTools/usr/bin/make -f CMakeFiles/copy_KeOps_formula.dir/build.make CMakeFiles/copy_KeOps_formula.dir/build [100%] Building CXX object CMakeFiles/copy_KeOps_formula.dir/Users/izzatum/opt/anaconda3/lib/python3.8/site-packages/pykeops/keops/core/link_autodiff.cpp.o /Library/Developer/CommandLineTools/usr/bin/c++ -DC_CONTIGUOUS=1 -DUSE_CUDA=0 -DUSE_DOUBLE=1 -DUSE_HALF=0 -DTYPE=double -I/Users/izzatum/opt/anaconda3/lib/python3.8/site-packages/pykeops/cmake_scripts/script_keops_formula/../.. -I/Users/izzatum/.cache/pykeops-1.5-cpython-38/build-cafbf708bb -I/Users/izzatum/opt/anaconda3/lib/python3.8/site-packages/pykeops/cmake_scripts/script_keops_formula/../../keops -ferror-limit=2 -g -O0 -g -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk -fPIC -include KeOps_formula.h -std=gnu++14 -MD -MT CMakeFiles/copy_KeOps_formula.dir/Users/izzatum/opt/anaconda3/lib/python3.8/site-packages/pykeops/keops/core/link_autodiff.cpp.o -MF CMakeFiles/copy_KeOps_formula.dir/Users/izzatum/opt/anaconda3/lib/python3.8/site-packages/pykeops/keops/core/link_autodiff.cpp.o.d -o CMakeFiles/copy_KeOps_formula.dir/Users/izzatum/opt/anaconda3/lib/python3.8/site-packages/pykeops/keops/core/link_autodiff.cpp.o -c /Users/izzatum/opt/anaconda3/lib/python3.8/site-packages/pykeops/keops/core/link_autodiff.cpp

[pyKeOps] Compiling pybind11 template libKeOps_template_ff040ce00d in /Users/izzatum/.cache/pykeops-1.5-cpython-38 ... Traceback (most recent call last): File "/Users/izzatum/opt/anaconda3/lib/python3.8/site-packages/pykeops/test/install.py", line 22, in test_numpy_bindings my_conv = pknp.Genred(formula, var) File "/Users/izzatum/opt/anaconda3/lib/python3.8/site-packages/pykeops/numpy/generic/generic_red.py", line 182, in init self.myconv = LoadKeOps( File "/Users/izzatum/opt/anaconda3/lib/python3.8/site-packages/pykeops/common/keops_io.py", line 48, in init self._safe_compile() File "/Users/izzatum/opt/anaconda3/lib/python3.8/site-packages/pykeops/common/utils.py", line 75, in wrapper_filelock func_res = func(*args, **kwargs) File "/Users/izzatum/opt/anaconda3/lib/python3.8/site-packages/pykeops/common/keops_io.py", line 55, in _safe_compile compile_generic_routine( File "/Users/izzatum/opt/anaconda3/lib/python3.8/site-packages/pykeops/common/compile_routines.py", line 245, in compile_generic_routine template_name, is_rebuilt = get_or_build_pybind11_template( File "/Users/izzatum/opt/anaconda3/lib/python3.8/site-packages/pykeops/common/compile_routines.py", line 86, in get_or_build_pybind11_template os.rename( FileNotFoundError: [Errno 2] No such file or directory: '/Users/izzatum/.cache/pykeops-1.5-cpython-38//KeOps_formula.o' -> '/Users/izzatum/.cache/pykeops-1.5-cpython-38//build-pybind11_template-libKeOps_template_16fe3306d9/KeOps_formula.o'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "", line 1, in File "/Users/izzatum/opt/anaconda3/lib/python3.8/site-packages/pykeops/test/install.py", line 35, in test_numpy_bindings my_conv = pknp.Genred(formula, var) File "/Users/izzatum/opt/anaconda3/lib/python3.8/site-packages/pykeops/numpy/generic/generic_red.py", line 182, in init self.myconv = LoadKeOps( File "/Users/izzatum/opt/anaconda3/lib/python3.8/site-packages/pykeops/common/keops_io.py", line 48, in init self._safe_compile() File "/Users/izzatum/opt/anaconda3/lib/python3.8/site-packages/pykeops/common/utils.py", line 75, in wrapper_filelock func_res = func(*args, **kwargs) File "/Users/izzatum/opt/anaconda3/lib/python3.8/site-packages/pykeops/common/keops_io.py", line 55, in _safe_compile compile_generic_routine( File "/Users/izzatum/opt/anaconda3/lib/python3.8/site-packages/pykeops/common/compile_routines.py", line 245, in compile_generic_routine template_name, is_rebuilt = get_or_build_pybind11_template( File "/Users/izzatum/opt/anaconda3/lib/python3.8/site-packages/pykeops/common/compile_routines.py", line 86, in get_or_build_pybind11_template os.rename( FileNotFoundError: [Errno 2] No such file or directory: '/Users/izzatum/.cache/pykeops-1.5-cpython-38//KeOps_formula.o' -> '/Users/izzatum/.cache/pykeops-1.5-cpython-38//build-pybind11_template-libKeOps_template_ff040ce00d/KeOps_formula.o' `

izzatum commented 2 years ago

Hi again! I have tried your colab version, and it works perfectly on Colab. However, on my own machines it doesn't work at all. Is there any difference between the colab version and the common pip install pykeops?

Thanks!

jeanfeydy commented 2 years ago

Hi @izzatum,

Thanks for your interest in the library and all my apologies for the long time to answer: Covid-December has been pretty disruptive for us in Paris. To answer your question, there is a small difference between pip install pykeops[colab] and the vanilla version (just a few dependencies which are added for convenience sake), but I do not believe that this is the issue here.

As discussed in #170 , could you maybe try our “beta” for KeOps v2.0 with:

pip install git+https://github.com/getkeops/keops.git@python_engine

And tell us if the problem is still there? There’s a good chance that PATH and configuration issues are handled better by our new compilation engine.

Best regards, Jean

izzatum commented 2 years ago

Hi @jeanfeydy,

Great thanks for your initiative for the improvements. I highly appreciate your initiatives and I love pykeops. I will give the beta version a try soon as I'm currently on holiday now. I will keep you posted.

Thanks!

Best regards, Izzat

izzatum commented 2 years ago

Hi @jeanfeydy,

I installed the git-master version of pykeops. However, I got the following error:

import pykeops Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/home/izzatum/anaconda3/lib/python3.9/site-packages/pykeops/__init__.py", line 31, in <module> import pykeops.config File "/home/izzatum/anaconda3/lib/python3.9/site-packages/pykeops/config.py", line 4, in <module> import keops.config File "/home/izzatum/anaconda3/lib/python3.9/site-packages/keops/__init__.py", line 11, in <module> with open(os.path.join(here, "keops_version"), encoding="utf-8") as v: FileNotFoundError: [Errno 2] No such file or directory: '/home/izzatum/anaconda3/lib/python3.9/site-packages/keops/keops_version

What would be the best solution to solve this? Thanks!