search

getkeops / keops

KErnel OPerationS, on CPUs and GPUs, with autodiff and without memory overflows
https://www.kernel-operations.io
MIT License
1.04k stars 64 forks source link

CPU-only installation with Apple clang 12 #178

Closed connor-duffin closed 3 years ago

connor-duffin commented 3 years ago

Hi guys,

Thanks for this great library. I am currently trying to install KeOps on an Intel Macbook Air 2020 (running tests locally before scaling up) but am coming unstuck with some errors in the compilation process.

I am installing in a clean conda environment which only has numpy and pykeops installed. I get: cmake --version returning cmake version 3.20.5, and clang++ --version returning Apple clang version 12.0.5 (clang-1205.0.22.11).

Running

import pykeops
pykeops.clean_pykeops() 
pykeops.test_numpy_bindings()

in a Python session then gives the following errors:

[pyKeOps]: Warning, no cuda detected. Switching to cpu only.
Cleaning /Users/connor/.cache/pykeops-1.5-cpython-39/...
    - /Users/connor/.cache/pykeops-1.5-cpython-39/build-pybind11_template-libKeOps_template_c65861668f has been removed.
    - /Users/connor/.cache/pykeops-1.5-cpython-39/build-5982a79418 has been removed.
    - /Users/connor/.cache/pykeops-1.5-cpython-39/keops_hash.log has been removed.
[pyKeOps] Initializing build folder for dtype=float64 and lang=numpy in /Users/connor/.cache/pykeops-1.5-cpython-39 ... done.
[pyKeOps] Compiling libKeOpsnumpy00df286daa in /Users/connor/.cache/pykeops-1.5-cpython-39:
       formula: Sum_Reduction(SqNorm2(x - y),1)
       aliases: x = Vi(0,3); y = Vj(1,3); 
       dtype  : float64
... 
In file included from <built-in>:1:
In file included from /Users/connor/.cache/pykeops-1.5-cpython-39/build-da8ca4eef9/KeOps_formula.h:13:
In file included from /Users/connor/miniconda3/envs/keops_small/lib/python3.9/site-packages/pykeops/cmake_scripts/script_keops_formula/../../keops/keops_includes.h:46:
In file included from /Users/connor/miniconda3/envs/keops_small/lib/python3.9/site-packages/pykeops/cmake_scripts/script_keops_formula/../../keops/core/formulas/constants/IntConst.h:3:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/sstream:173:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/ostream:138:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/ios:215:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/__locale:14:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/string:506:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/string_view:175:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/__string:57:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/algorithm:639:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/initializer_list:46:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/cstddef:37:
/Users/connor/miniconda3/envs/keops_small/lib/python3.9/site-packages/pykeops/cmake_scripts/script_keops_formula/../../version:1:1: error: expected unqualified-id
1.5
^
In file included from <built-in>:1:
In file included from /Users/connor/.cache/pykeops-1.5-cpython-39/build-da8ca4eef9/KeOps_formula.h:13:
In file included from /Users/connor/miniconda3/envs/keops_small/lib/python3.9/site-packages/pykeops/cmake_scripts/script_keops_formula/../../keops/keops_includes.h:46:
In file included from /Users/connor/miniconda3/envs/keops_small/lib/python3.9/site-packages/pykeops/cmake_scripts/script_keops_formula/../../keops/core/formulas/constants/IntConst.h:3:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/sstream:173:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/ostream:138:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/ios:215:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/__locale:14:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/string:506:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/string_view:175:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/__string:57:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/algorithm:640:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/type_traits:418:
/Users/connor/miniconda3/envs/keops_small/lib/python3.9/site-packages/pykeops/cmake_scripts/script_keops_formula/../../version:1:1: error: expected unqualified-id
1.5
^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
3 errors generated.
make[3]: *** [CMakeFiles/copy_KeOps_formula.dir/Users/connor/miniconda3/envs/keops_small/lib/python3.9/site-packages/pykeops/keops/core/link_autodiff.cpp.o] Error 1
make[2]: *** [CMakeFiles/copy_KeOps_formula.dir/all] Error 2
make[1]: *** [CMakeFiles/KeOps_formula.dir/rule] Error 2
make: *** [KeOps_formula] Error 2

And the Make debug:

--------------------- MAKE DEBUG -----------------
Command '['cmake', '--build', '.', '--target', 'KeOps_formula', '--', 'VERBOSE=1']' returned non-zero exit status 2.
/usr/local/Cellar/cmake/3.20.5/bin/cmake -S/Users/connor/miniconda3/envs/keops_small/lib/python3.9/site-packages/pykeops/cmake_scripts/script_keops_formula -B/Users/connor/.cache/pykeops-1.5-cpython-39/build-da8ca4eef9 --check-build-system CMakeFiles/Makefile.cmake 0
/Applications/Xcode.app/Contents/Developer/usr/bin/make  -f CMakeFiles/Makefile2 KeOps_formula
/usr/local/Cellar/cmake/3.20.5/bin/cmake -S/Users/connor/miniconda3/envs/keops_small/lib/python3.9/site-packages/pykeops/cmake_scripts/script_keops_formula -B/Users/connor/.cache/pykeops-1.5-cpython-39/build-da8ca4eef9 --check-build-system CMakeFiles/Makefile.cmake 0
/usr/local/Cellar/cmake/3.20.5/bin/cmake -E cmake_progress_start /Users/connor/.cache/pykeops-1.5-cpython-39/build-da8ca4eef9/CMakeFiles 1
/Applications/Xcode.app/Contents/Developer/usr/bin/make  -f CMakeFiles/Makefile2 CMakeFiles/KeOps_formula.dir/all
/Applications/Xcode.app/Contents/Developer/usr/bin/make  -f CMakeFiles/copy_KeOps_formula.dir/build.make CMakeFiles/copy_KeOps_formula.dir/depend
cd /Users/connor/.cache/pykeops-1.5-cpython-39/build-da8ca4eef9 && /usr/local/Cellar/cmake/3.20.5/bin/cmake -E cmake_depends "Unix Makefiles" /Users/connor/miniconda3/envs/keops_small/lib/python3.9/site-packages/pykeops/cmake_scripts/script_keops_formula /Users/connor/miniconda3/envs/keops_small/lib/python3.9/site-packages/pykeops/cmake_scripts/script_keops_formula /Users/connor/.cache/pykeops-1.5-cpython-39/build-da8ca4eef9 /Users/connor/.cache/pykeops-1.5-cpython-39/build-da8ca4eef9 /Users/connor/.cache/pykeops-1.5-cpython-39/build-da8ca4eef9/CMakeFiles/copy_KeOps_formula.dir/DependInfo.cmake --color=
/Applications/Xcode.app/Contents/Developer/usr/bin/make  -f CMakeFiles/copy_KeOps_formula.dir/build.make CMakeFiles/copy_KeOps_formula.dir/build
[100%] Building CXX object CMakeFiles/copy_KeOps_formula.dir/Users/connor/miniconda3/envs/keops_small/lib/python3.9/site-packages/pykeops/keops/core/link_autodiff.cpp.o
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -DC_CONTIGUOUS=1 -DUSE_CUDA=0 -DUSE_DOUBLE=1 -DUSE_HALF=0 -D__TYPE__=double -I/Users/connor/miniconda3/envs/keops_small/lib/python3.9/site-packages/pykeops/cmake_scripts/script_keops_formula/../.. -I/Users/connor/.cache/pykeops-1.5-cpython-39/build-da8ca4eef9 -I/Users/connor/miniconda3/envs/keops_small/lib/python3.9/site-packages/pykeops/cmake_scripts/script_keops_formula/../../keops -ferror-limit=2 -O3 -DNDEBUG -O3 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.3.sdk -fPIC -include KeOps_formula.h -std=gnu++14 -MD -MT CMakeFiles/copy_KeOps_formula.dir/Users/connor/miniconda3/envs/keops_small/lib/python3.9/site-packages/pykeops/keops/core/link_autodiff.cpp.o -MF CMakeFiles/copy_KeOps_formula.dir/Users/connor/miniconda3/envs/keops_small/lib/python3.9/site-packages/pykeops/keops/core/link_autodiff.cpp.o.d -o CMakeFiles/copy_KeOps_formula.dir/Users/connor/miniconda3/envs/keops_small/lib/python3.9/site-packages/pykeops/keops/core/link_autodiff.cpp.o -c /Users/connor/miniconda3/envs/keops_small/lib/python3.9/site-packages/pykeops/keops/core/link_autodiff.cpp

--------------------- ----------- -----------------
[pyKeOps] Compiling pybind11 template libKeOps_template_0f04bb694f in /Users/connor/.cache/pykeops-1.5-cpython-39 ... [pyKeOps] Initializing build folder for dtype=float64 and lang=numpy in /Users/connor/.cache/pykeops-1.5-cpython-39 ... -- The CXX compiler identification is AppleClang 12.0.5.12050022
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Looking for a CUDA compiler
-- Looking for a CUDA compiler - NOTFOUND
-- No CUDA support
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/connor/.cache/pykeops-1.5-cpython-39/build-5982a79418

done.
[pyKeOps] Compiling libKeOpsnumpy00df286daa in /Users/connor/.cache/pykeops-1.5-cpython-39:
       formula: Sum_Reduction(SqNorm2(x - y),1)
       aliases: x = Vi(0,3); y = Vj(1,3); 
       dtype  : float64
... 
In file included from <built-in>:1:
In file included from /Users/connor/.cache/pykeops-1.5-cpython-39/build-5982a79418/KeOps_formula.h:13:
In file included from /Users/connor/miniconda3/envs/keops_small/lib/python3.9/site-packages/pykeops/cmake_scripts/script_keops_formula/../../keops/keops_includes.h:46:
In file included from /Users/connor/miniconda3/envs/keops_small/lib/python3.9/site-packages/pykeops/cmake_scripts/script_keops_formula/../../keops/core/formulas/constants/IntConst.h:3:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/sstream:173:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/ostream:138:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/ios:215:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/__locale:14:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/string:506:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/string_view:175:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/__string:57:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/algorithm:639:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/initializer_list:46:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/cstddef:37:
/Users/connor/miniconda3/envs/keops_small/lib/python3.9/site-packages/pykeops/cmake_scripts/script_keops_formula/../../version:1:1: error: expected unqualified-id
1.5
^
In file included from <built-in>:1:
In file included from /Users/connor/.cache/pykeops-1.5-cpython-39/build-5982a79418/KeOps_formula.h:13:
In file included from /Users/connor/miniconda3/envs/keops_small/lib/python3.9/site-packages/pykeops/cmake_scripts/script_keops_formula/../../keops/keops_includes.h:46:
In file included from /Users/connor/miniconda3/envs/keops_small/lib/python3.9/site-packages/pykeops/cmake_scripts/script_keops_formula/../../keops/core/formulas/constants/IntConst.h:3:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/sstream:173:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/ostream:138:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/ios:215:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/__locale:14:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/string:506:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/string_view:175:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/__string:57:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/algorithm:640:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.3.sdk/usr/include/c++/v1/type_traits:418:
/Users/connor/miniconda3/envs/keops_small/lib/python3.9/site-packages/pykeops/cmake_scripts/script_keops_formula/../../version:1:1: error: expected unqualified-id
1.5
^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
3 errors generated.
make[3]: *** [CMakeFiles/copy_KeOps_formula.dir/Users/connor/miniconda3/envs/keops_small/lib/python3.9/site-packages/pykeops/keops/core/link_autodiff.cpp.o] Error 1
make[2]: *** [CMakeFiles/copy_KeOps_formula.dir/all] Error 2
make[1]: *** [CMakeFiles/KeOps_formula.dir/rule] Error 2
make: *** [KeOps_formula] Error 2

Any advice would be much appreciated.

Best, Connor

hypnopump commented 3 years ago

Can you try the export CUDA_VISIBLE_DEVICES= before running that script?

connor-duffin commented 3 years ago

Hi, just tried this, and with export CUDA_VISIBLE_DEVICES=, it still throws the above error.

joanglaunes commented 3 years ago

Hello @connor-duffin , This was a stupid bug, a name conflict between files named simply version in the Apple clang SDK and our root directory. I have done the fix in the master branch now. The fix was just renaming this file to keops_version and change line 7 of init.py file accordingly.

connor-duffin commented 3 years ago

Hey @joanglaunes, sorry for the delay, but just checked and this has indeed fixed it. Thanks!