search

FalkonML / falkon

Large-scale, multi-GPU capable, kernel solver
https://falkonml.github.io/falkon/
MIT License
181 stars 22 forks source link

Import falkon gives the following error, any suggestions what is missing as I just followed the steps from the installation ==> #2

Closed Giodiro closed 4 years ago

Giodiro commented 4 years ago

Import falkon gives the following error, any suggestions what is missing as I just followed the steps from the installation ==>

sparse_helpers.so: undefined symbol: _ZN5torch3jit6tracer9addOutputEPNS0_4NodeERKN2at6TensorE

Originally posted by @jaiabhayk in https://github.com/FalkonML/falkon/issues/1#issuecomment-650699506

Giodiro commented 4 years ago

Hi @jaiabhayk, this error can have a few causes:

  1. Try to install pytorch via anaconda. It seems to be more stable.
  2. Check the version of gcc you have installed. You need gcc >= 4.9 to compile torch extensions (for example sparse_helpers.so)

Giacomo

jaiabhayk commented 4 years ago

I used gcc version 6.3.0 (GCC), is that fine ?

Giodiro commented 4 years ago

That should be fine. If you can, please try to install falkon in a "cleanish" environment:

conda create -n falkon
conda install pytorch torchvision cudatoolkit=10.2 -c pytorch
conda install Cython
git clone --recurse-submodules https://github.com/FalkonML/falkon.git
cd falkon
pip install ./keops
pip install .

Note that you may have to change the second line, depending on your system (this will work to install the latest version of pytorch with the 10.2 version of cuda).

Let me know if this works, or if there are errors.

jaiabhayk commented 4 years ago

I am installing it and will let you know, thanks a lot for your help.

jaiabhayk commented 4 years ago

gcc/7.5.0 cython-0.29.20 Successfully installed GPUtil-1.4.0 pykeops-1.4.1.

import falkon gives the following error, do you need latest anconda or some other latest package, also I am using cuda/10.1.243, that should be fine right ? ==>

from falkon.sparse.sparse_helpers import normsq, norm ModuleNotFoundError: No module named 'falkon.sparse.sparse_helpers'

jaiabhayk commented 4 years ago

After install, did you try ? import falkon

Giodiro commented 4 years ago

Hi @jaiabhayk The cuda version is fine, but there must be some error when you run pip install . inside the falkon directory. The ModuleNotFoundError indicates that one of the pytorch extensions (falkon.sparse.sparse_helpers) was not installed correctly. Could you attach the output of running python setup.py develop from within the root directory of this repository?

Sorry that the installation process is still a bit complex!

jaiabhayk commented 4 years ago 
(falkon5) bash-4.2$ python setup.py develop
which: no hipcc in (/scratch0/User/installs/anconda37Latest/envs/falkon5/bin:/scratch0/User/installs/anconda37Latest/condabin:/BaseDir/jaiUser/torch/install/bin:/BaseDir/jaiUser/torch/install/bin:/BaseDir/jaiUser/torch/install/bin:/opt/local/stow/cmake-3.16.2/bin:/scratch0/User/installs/anconda37Latest/bin:/opt/local/stow/gcc-7.5.0/bin:/opt/local/stow/autogen-5.18.7/bin:/opt/local/stow/dejagnu-1.6/bin:/opt/local/stow/cloog-0.18.4/bin:/opt/local/stow/ppl-1.2/bin:/opt/common/cuda/cuda-10.2.89/bin:/BaseDir/jaiUser/torch/install/bin:/BaseDir/jaiUser/torch/install/bin:/BaseDir/jaiUser/torch/install/bin:/BaseDir/jaiUser/torch/install/bin:/BaseDir/jaiUser/torch/install/bin:/BaseDir/jaiUser/torch/install/bin:/opt/local/bin:/usr/lib64/qt-3.3/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin)
Compiling falkon/utils/cyblas.pyx because it changed.
[1/1] Cythonizing falkon/utils/cyblas.pyx
running develop
running egg_info
creating falkon.egg-info
writing falkon.egg-info/PKG-INFO
writing dependency_links to falkon.egg-info/dependency_links.txt
writing requirements to falkon.egg-info/requires.txt
writing top-level names to falkon.egg-info/top_level.txt
writing manifest file 'falkon.egg-info/SOURCES.txt'
reading manifest file 'falkon.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no previously-included files matching '__pycache__' found anywhere in distribution
warning: no previously-included files matching '*.py[co]' found anywhere in distribution
writing manifest file 'falkon.egg-info/SOURCES.txt'
running build_ext
building 'falkon.sparse.sparse_helpers' extension
creating build
creating build/temp.linux-x86_64-3.8
creating build/temp.linux-x86_64-3.8/falkon
creating build/temp.linux-x86_64-3.8/falkon/sparse
creating build/temp.linux-x86_64-3.8/falkon/sparse/cpp
creating build/temp.linux-x86_64-3.8/falkon/sparse/cuda
/opt/local/stow/gcc-7.5.0/bin/gcc -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/opt/local/stow/isl-0.16.1/include -I/opt/local/stow/mpc-1.0.3/include -I/opt/local/stow/mpfr-3.1.4/include -I/opt/local/stow/gmp-6.1.1/include -fPIC -DWITH_CUDA -I./falkon/sparse -I/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include -I/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -I/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/TH -I/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/THC -I/opt/common/cuda/cuda-10.2.89/include -I/scratch0/User/installs/anconda37Latest/envs/falkon5/include/python3.8 -c ./falkon/sparse/sparse_extension.cpp -o build/temp.linux-x86_64-3.8/./falkon/sparse/sparse_extension.o -fopenmp -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=sparse_helpers -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
/opt/local/stow/gcc-7.5.0/bin/gcc -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/opt/local/stow/isl-0.16.1/include -I/opt/local/stow/mpc-1.0.3/include -I/opt/local/stow/mpfr-3.1.4/include -I/opt/local/stow/gmp-6.1.1/include -fPIC -DWITH_CUDA -I./falkon/sparse -I/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include -I/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -I/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/TH -I/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/THC -I/opt/common/cuda/cuda-10.2.89/include -I/scratch0/User/installs/anconda37Latest/envs/falkon5/include/python3.8 -c ./falkon/sparse/cpp/sparse_matmul.cpp -o build/temp.linux-x86_64-3.8/./falkon/sparse/cpp/sparse_matmul.o -fopenmp -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=sparse_helpers -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
/opt/local/stow/gcc-7.5.0/bin/gcc -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/opt/local/stow/isl-0.16.1/include -I/opt/local/stow/mpc-1.0.3/include -I/opt/local/stow/mpfr-3.1.4/include -I/opt/local/stow/gmp-6.1.1/include -fPIC -DWITH_CUDA -I./falkon/sparse -I/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include -I/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -I/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/TH -I/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/THC -I/opt/common/cuda/cuda-10.2.89/include -I/scratch0/User/installs/anconda37Latest/envs/falkon5/include/python3.8 -c ./falkon/sparse/cpp/sparse_norm.cpp -o build/temp.linux-x86_64-3.8/./falkon/sparse/cpp/sparse_norm.o -fopenmp -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=sparse_helpers -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
/opt/common/cuda/cuda-10.2.89/bin/nvcc -DWITH_CUDA -I./falkon/sparse -I/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include -I/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -I/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/TH -I/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/THC -I/opt/common/cuda/cuda-10.2.89/include -I/scratch0/User/installs/anconda37Latest/envs/falkon5/include/python3.8 -c ./falkon/sparse/cuda/csr2dense_cuda.cu -o build/temp.linux-x86_64-3.8/./falkon/sparse/cuda/csr2dense_cuda.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options '-fPIC' --expt-relaxed-constexpr -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=sparse_helpers -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_61,code=sm_61 -std=c++14
/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(14): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(15): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(15): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(15): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(18): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(19): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(19): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(19): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(23): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(24): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(24): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(24): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/autograd/profiler.h(100): warning: attribute "__visibility__" does not apply here

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/autograd/profiler.h(115): warning: attribute "__visibility__" does not apply here

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(14): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(15): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(15): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(15): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(18): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(19): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(19): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(19): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(23): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(24): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(24): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(24): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/autograd/profiler.h(100): warning: attribute "__visibility__" does not apply here

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/autograd/profiler.h(115): warning: attribute "__visibility__" does not apply here

/opt/common/cuda/cuda-10.2.89/bin/nvcc -DWITH_CUDA -I./falkon/sparse -I/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include -I/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -I/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/TH -I/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/THC -I/opt/common/cuda/cuda-10.2.89/include -I/scratch0/User/installs/anconda37Latest/envs/falkon5/include/python3.8 -c ./falkon/sparse/cuda/spspmm_cuda.cu -o build/temp.linux-x86_64-3.8/./falkon/sparse/cuda/spspmm_cuda.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options '-fPIC' --expt-relaxed-constexpr -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=sparse_helpers -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_61,code=sm_61 -std=c++14
/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(14): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(15): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(15): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(15): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(18): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(19): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(19): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(19): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(23): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(24): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(24): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(24): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/autograd/profiler.h(100): warning: attribute "__visibility__" does not apply here

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/autograd/profiler.h(115): warning: attribute "__visibility__" does not apply here

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(14): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(15): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(15): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(15): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(18): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(19): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(19): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(19): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(23): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(24): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(24): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(24): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/autograd/profiler.h(100): warning: attribute "__visibility__" does not apply here

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/autograd/profiler.h(115): warning: attribute "__visibility__" does not apply here

creating build/lib.linux-x86_64-3.8
creating build/lib.linux-x86_64-3.8/falkon
creating build/lib.linux-x86_64-3.8/falkon/sparse
/opt/local/stow/gcc-7.5.0/bin/g++ -pthread -shared -B /scratch0/User/installs/anconda37Latest/envs/falkon5/compiler_compat -L/scratch0/User/installs/anconda37Latest/envs/falkon5/lib -Wl,-rpath=/scratch0/User/installs/anconda37Latest/envs/falkon5/lib -Wl,--no-as-needed -Wl,--sysroot=/ -L/opt/local/stow/isl-0.16.1/lib -L/opt/local/stow/mpc-1.0.3/lib -L/opt/local/stow/mpfr-3.1.4/lib -L/opt/local/stow/gmp-6.1.1/lib -I/opt/local/stow/isl-0.16.1/include -I/opt/local/stow/mpc-1.0.3/include -I/opt/local/stow/mpfr-3.1.4/include -I/opt/local/stow/gmp-6.1.1/include build/temp.linux-x86_64-3.8/./falkon/sparse/sparse_extension.o build/temp.linux-x86_64-3.8/./falkon/sparse/cpp/sparse_matmul.o build/temp.linux-x86_64-3.8/./falkon/sparse/cpp/sparse_norm.o build/temp.linux-x86_64-3.8/./falkon/sparse/cuda/csr2dense_cuda.o build/temp.linux-x86_64-3.8/./falkon/sparse/cuda/spspmm_cuda.o -L/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/lib -L/opt/common/cuda/cuda-10.2.89/lib64 -lc10 -ltorch -ltorch_cpu -ltorch_python -lcudart -lc10_cuda -ltorch_cuda -o build/lib.linux-x86_64-3.8/falkon/sparse/sparse_helpers.so -lcusparse -l cusparse
building 'falkon.ooc_ops.multigpu_potrf' extension
creating build/temp.linux-x86_64-3.8/falkon/ooc_ops
creating build/temp.linux-x86_64-3.8/falkon/ooc_ops/multigpu
creating build/temp.linux-x86_64-3.8/falkon/ooc_ops/multigpu/cuda
/opt/local/stow/gcc-7.5.0/bin/gcc -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/opt/local/stow/isl-0.16.1/include -I/opt/local/stow/mpc-1.0.3/include -I/opt/local/stow/mpfr-3.1.4/include -I/opt/local/stow/gmp-6.1.1/include -fPIC -DWITH_CUDA -I./falkon/ooc_ops/multigpu -I/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include -I/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -I/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/TH -I/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/THC -I/opt/common/cuda/cuda-10.2.89/include -I/scratch0/User/installs/anconda37Latest/envs/falkon5/include/python3.8 -c ./falkon/ooc_ops/multigpu/multigpu_potrf_bind.cpp -o build/temp.linux-x86_64-3.8/./falkon/ooc_ops/multigpu/multigpu_potrf_bind.o -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=multigpu_potrf -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
/opt/common/cuda/cuda-10.2.89/bin/nvcc -DWITH_CUDA -I./falkon/ooc_ops/multigpu -I/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include -I/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -I/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/TH -I/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/THC -I/opt/common/cuda/cuda-10.2.89/include -I/scratch0/User/installs/anconda37Latest/envs/falkon5/include/python3.8 -c ./falkon/ooc_ops/multigpu/cuda/multigpu_potrf.cu -o build/temp.linux-x86_64-3.8/./falkon/ooc_ops/multigpu/cuda/multigpu_potrf.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options '-fPIC' --expt-relaxed-constexpr -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=multigpu_potrf -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_61,code=sm_61 -std=c++14
/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(14): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(15): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(15): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(15): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(18): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(19): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(19): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(19): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(23): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(24): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(24): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(24): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/autograd/profiler.h(100): warning: attribute "__visibility__" does not apply here

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/autograd/profiler.h(115): warning: attribute "__visibility__" does not apply here

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(14): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(15): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(15): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(15): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(18): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(19): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(19): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(19): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(23): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(24): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(24): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/api/include/torch/nn/functional/padding.h(24): warning: integer conversion resulted in a change of sign

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/autograd/profiler.h(100): warning: attribute "__visibility__" does not apply here

/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/include/torch/csrc/autograd/profiler.h(115): warning: attribute "__visibility__" does not apply here

creating build/lib.linux-x86_64-3.8/falkon/ooc_ops
/opt/local/stow/gcc-7.5.0/bin/g++ -pthread -shared -B /scratch0/User/installs/anconda37Latest/envs/falkon5/compiler_compat -L/scratch0/User/installs/anconda37Latest/envs/falkon5/lib -Wl,-rpath=/scratch0/User/installs/anconda37Latest/envs/falkon5/lib -Wl,--no-as-needed -Wl,--sysroot=/ -L/opt/local/stow/isl-0.16.1/lib -L/opt/local/stow/mpc-1.0.3/lib -L/opt/local/stow/mpfr-3.1.4/lib -L/opt/local/stow/gmp-6.1.1/lib -I/opt/local/stow/isl-0.16.1/include -I/opt/local/stow/mpc-1.0.3/include -I/opt/local/stow/mpfr-3.1.4/include -I/opt/local/stow/gmp-6.1.1/include build/temp.linux-x86_64-3.8/./falkon/ooc_ops/multigpu/multigpu_potrf_bind.o build/temp.linux-x86_64-3.8/./falkon/ooc_ops/multigpu/cuda/multigpu_potrf.o -L/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/torch/lib -L/opt/common/cuda/cuda-10.2.89/lib64 -lcusolver -lcublas -lc10 -ltorch -ltorch_cpu -ltorch_python -lcudart -lc10_cuda -ltorch_cuda -o build/lib.linux-x86_64-3.8/falkon/ooc_ops/multigpu_potrf.so -lcublas -l cublas -lcusolver -l cusolver
building 'falkon.utils.cyblas' extension
creating build/temp.linux-x86_64-3.8/falkon/utils
/opt/local/stow/gcc-7.5.0/bin/gcc -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/opt/local/stow/isl-0.16.1/include -I/opt/local/stow/mpc-1.0.3/include -I/opt/local/stow/mpfr-3.1.4/include -I/opt/local/stow/gmp-6.1.1/include -fPIC -I/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/numpy/core/include -I/scratch0/User/installs/anconda37Latest/envs/falkon5/include/python3.8 -c falkon/utils/cyblas.c -o build/temp.linux-x86_64-3.8/falkon/utils/cyblas.o -shared -fPIC -fopenmp -O3 -Wall -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=cyblas -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++14
cc1: warning: command line option ‘-std=c++14’ is valid for C++/ObjC++ but not for C
In file included from /scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/numpy/core/include/numpy/ndarraytypes.h:1832:0,
                 from /scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/numpy/core/include/numpy/ndarrayobject.h:12,
                 from /scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/numpy/core/include/numpy/arrayobject.h:4,
                 from falkon/utils/cyblas.c:637:
/scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: #warning "Using deprecated NumPy API, disable it with " "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
 #warning "Using deprecated NumPy API, disable it with " \
  ^~~~~~~
creating build/lib.linux-x86_64-3.8/falkon/utils
gcc -pthread -shared -B /scratch0/User/installs/anconda37Latest/envs/falkon5/compiler_compat -L/scratch0/User/installs/anconda37Latest/envs/falkon5/lib -Wl,-rpath=/scratch0/User/installs/anconda37Latest/envs/falkon5/lib -Wl,--no-as-needed -Wl,--sysroot=/ -L/opt/local/stow/isl-0.16.1/lib -L/opt/local/stow/mpc-1.0.3/lib -L/opt/local/stow/mpfr-3.1.4/lib -L/opt/local/stow/gmp-6.1.1/lib -I/opt/local/stow/isl-0.16.1/include -I/opt/local/stow/mpc-1.0.3/include -I/opt/local/stow/mpfr-3.1.4/include -I/opt/local/stow/gmp-6.1.1/include build/temp.linux-x86_64-3.8/falkon/utils/cyblas.o -o build/lib.linux-x86_64-3.8/falkon/utils/cyblas.so -fPIC -fopenmp -s
copying build/lib.linux-x86_64-3.8/falkon/sparse/sparse_helpers.so -> falkon/sparse
copying build/lib.linux-x86_64-3.8/falkon/ooc_ops/multigpu_potrf.so -> falkon/ooc_ops
copying build/lib.linux-x86_64-3.8/falkon/utils/cyblas.so -> falkon/utils
Creating /scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/falkon.egg-link (link to .)
Adding falkon 0.5.0 to easy-install.pth file

Installed /scratch1/User/Users/JaiUser/Documents/Study/research/falkon_6/falkon
Processing dependencies for falkon==0.5.0
Searching for psutil
Reading https://pypi.org/simple/psutil/
Downloading https://files.pythonhosted.org/packages/c4/b8/3512f0e93e0db23a71d82485ba256071ebef99b227351f0f5540f744af41/psutil-5.7.0.tar.gz#sha256=685ec16ca14d079455892f25bd124df26ff9137664af445563c1bd36629b5e0e
Best match: psutil 5.7.0
Processing psutil-5.7.0.tar.gz
Writing /tmp/easy_install-_1ygdkof/psutil-5.7.0/setup.cfg
Running psutil-5.7.0/setup.py -q bdist_egg --dist-dir /tmp/easy_install-_1ygdkof/psutil-5.7.0/egg-dist-tmp-bxoc5x8o
creating /scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/psutil-5.7.0-py3.8-linux-x86_64.egg
Extracting psutil-5.7.0-py3.8-linux-x86_64.egg to /scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages
Adding psutil 5.7.0 to easy-install.pth file

Installed /scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/psutil-5.7.0-py3.8-linux-x86_64.egg
Searching for scikit-learn
Reading https://pypi.org/simple/scikit-learn/
Downloading https://files.pythonhosted.org/packages/47/14/c094698b7dd17cd2e289974a78d6d2df78c0d9eb0ac4d8d5fad255aaf977/scikit_learn-0.23.1-cp38-cp38-manylinux1_x86_64.whl#sha256=3e6e92b495eee193a8fa12a230c9b7976ea0fc1263719338e35c986ea1e42cff
Best match: scikit-learn 0.23.1
Processing scikit_learn-0.23.1-cp38-cp38-manylinux1_x86_64.whl
Installing scikit_learn-0.23.1-cp38-cp38-manylinux1_x86_64.whl to /scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages
Adding scikit-learn 0.23.1 to easy-install.pth file

Installed /scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/scikit_learn-0.23.1-py3.8-linux-x86_64.egg
Searching for threadpoolctl>=2.0.0
Reading https://pypi.org/simple/threadpoolctl/
Downloading https://files.pythonhosted.org/packages/f7/12/ec3f2e203afa394a149911729357aa48affc59c20e2c1c8297a60f33f133/threadpoolctl-2.1.0-py3-none-any.whl#sha256=38b74ca20ff3bb42caca8b00055111d74159ee95c4370882bbff2b93d24da725
Best match: threadpoolctl 2.1.0
Processing threadpoolctl-2.1.0-py3-none-any.whl
Installing threadpoolctl-2.1.0-py3-none-any.whl to /scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages
Adding threadpoolctl 2.1.0 to easy-install.pth file

Installed /scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/threadpoolctl-2.1.0-py3.8.egg
Searching for joblib>=0.11
Reading https://pypi.org/simple/joblib/
Downloading https://files.pythonhosted.org/packages/51/dd/0e015051b4a27ec5a58b02ab774059f3289a94b0906f880a3f9507e74f38/joblib-0.16.0-py3-none-any.whl#sha256=d348c5d4ae31496b2aa060d6d9b787864dd204f9480baaa52d18850cb43e9f49
Best match: joblib 0.16.0
Processing joblib-0.16.0-py3-none-any.whl
Installing joblib-0.16.0-py3-none-any.whl to /scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages
Adding joblib 0.16.0 to easy-install.pth file

Installed /scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/joblib-0.16.0-py3.8.egg
Searching for future
Reading https://pypi.org/simple/future/
Downloading https://files.pythonhosted.org/packages/45/0b/38b06fd9b92dc2b68d58b75f900e97884c45bedd2ff83203d933cf5851c9/future-0.18.2.tar.gz#sha256=b1bead90b70cf6ec3f0710ae53a525360fa360d306a86583adc6bf83a4db537d
Best match: future 0.18.2
Processing future-0.18.2.tar.gz
Writing /tmp/easy_install-t538_hut/future-0.18.2/setup.cfg
Running future-0.18.2/setup.py -q bdist_egg --dist-dir /tmp/easy_install-t538_hut/future-0.18.2/egg-dist-tmp-u506u24s
warning: no files found matching '*.au' under directory 'tests'
warning: no files found matching '*.gif' under directory 'tests'
warning: no files found matching '*.txt' under directory 'tests'
zip_safe flag not set; analyzing archive contents...
future.backports.test.__pycache__.ssl_servers.cpython-38: module references __file__
future.backports.test.__pycache__.support.cpython-38: module references __file__
future.standard_library.__pycache__.__init__.cpython-38: module references __file__
future.standard_library.__pycache__.__init__.cpython-38: module references __path__
future.utils.__pycache__.__init__.cpython-38: module MAY be using inspect.stack
past.builtins.__pycache__.misc.cpython-38: module MAY be using inspect.stack
past.translation.__pycache__.__init__.cpython-38: module references __file__
past.translation.__pycache__.__init__.cpython-38: module references __path__
creating /scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/future-0.18.2-py3.8.egg
Extracting future-0.18.2-py3.8.egg to /scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages
Adding future 0.18.2 to easy-install.pth file
Installing futurize script to /scratch0/User/installs/anconda37Latest/envs/falkon5/bin
Installing pasteurize script to /scratch0/User/installs/anconda37Latest/envs/falkon5/bin

Installed /scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages/future-0.18.2-py3.8.egg
Searching for numpy==1.18.5
Best match: numpy 1.18.5
Adding numpy 1.18.5 to easy-install.pth file
Installing f2py script to /scratch0/User/installs/anconda37Latest/envs/falkon5/bin
Installing f2py3 script to /scratch0/User/installs/anconda37Latest/envs/falkon5/bin
Installing f2py3.8 script to /scratch0/User/installs/anconda37Latest/envs/falkon5/bin

Using /scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages
Searching for scipy==1.5.1
Best match: scipy 1.5.1
Adding scipy 1.5.1 to easy-install.pth file

Using /scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages
Searching for torch==1.5.1
Best match: torch 1.5.1
Adding torch 1.5.1 to easy-install.pth file
Installing convert-caffe2-to-onnx script to /scratch0/User/installs/anconda37Latest/envs/falkon5/bin
Installing convert-onnx-to-caffe2 script to /scratch0/User/installs/anconda37Latest/envs/falkon5/bin

Using /scratch0/User/installs/anconda37Latest/envs/falkon5/lib/python3.8/site-packages
Finished processing dependencies for falkon==0.5.0
(falkon5) bash-4.2$
jaiabhayk commented 4 years ago

The "import falkon" works now but needs to use "python setup.py develop" before "pip install ." Thanks for your help.

jaiabhayk commented 4 years ago

It was also missing some additional steps, so just adding this to be complete commands.

1100 conda create -n falkon6 1101 conda activate falkon6 1102 conda install pytorch torchvision cudatoolkit=10.2 -c pytorch 1103 conda install Cython 1104 conda install scipy 1115 mkdir falkon_python_latest 1116 cd falkon_python_latest/ 1117 git clone --recurse-submodules https://github.com/FalkonML/falkon.git 1118 cd falkon 1119 pip install ./keops 1120 python setup.py develop 1122 pip install .

Giodiro commented 4 years ago

Great! It's weird that pip install . did not do the trick, but I'm glad it's working now :)