Can SparseConvNet be installed in user space ?

[samfux84]

I am trying to install SparseConvNet on the hpc cluster of our university.

It fails because I don't have write permission in the installation directory of Python:

The following error occurred while trying to add or remove files in the
installation directory:

    [Errno 13] Permission denied: '/cluster/apps/python/3.7.1/x86_64/lib64/python3.7/site-packages/test-easy-install-27744.write-test'

With pip, local installations are straight forward. It is sufficient to add the --user option and then the package is installed in $HOME/.local instead of the Python installation directory. Is something similar with SparseConvNet ?

Should

python setup.py develop --user

work ?

[samfux84]

When tryint to add --user in develop.sh, then compilation seems to work but then the test fails with a segmentation fault:

[sfux@lo-s4-040 SparseConvNet]$ bash develop.sh 
running develop
running egg_info
creating sparseconvnet.egg-info
writing sparseconvnet.egg-info/PKG-INFO
writing dependency_links to sparseconvnet.egg-info/dependency_links.txt
writing top-level names to sparseconvnet.egg-info/top_level.txt
writing manifest file 'sparseconvnet.egg-info/SOURCES.txt'
package init file 'sparseconvnet/SCN/__init__.py' not found (or not a regular file)
reading manifest file 'sparseconvnet.egg-info/SOURCES.txt'
writing manifest file 'sparseconvnet.egg-info/SOURCES.txt'
running build_ext
/cluster/apps/python/3.7.1/x86_64/lib64/python3.7/site-packages/torch/utils/cpp_extension.py:191: UserWarning: 

                               !! WARNING !!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Your compiler (/usr/bin/g++ 4.8.5) may be ABI-incompatible with PyTorch!
Please use a compiler that is ABI-compatible with GCC 4.9 and above.
See https://gcc.gnu.org/onlinedocs/libstdc++/manual/abi.html.

See https://gist.github.com/goldsborough/d466f43e8ffc948ff92de7486c5216d6
for instructions on how to install GCC 4.9 or higher.
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

                              !! WARNING !!

  warnings.warn(ABI_INCOMPATIBILITY_WARNING.format(compiler))
building 'sparseconvnet.SCN' extension
creating build
creating build/temp.linux-x86_64-3.7
creating build/temp.linux-x86_64-3.7/sparseconvnet
creating build/temp.linux-x86_64-3.7/sparseconvnet/SCN
/cluster/apps/gcc-4.8.5/cuda-10.0.130-y7x264d5jchmrxfbsqt2ztwerqrrwzru/bin/nvcc -I/cluster/apps/python/3.7.1/x86_64/include -I/cluster/home/sfux/test/hanxue/SparseConvNet/sparseconvnet/SCN/ -I/cluster/apps/python/3.7.1/x86_64/lib64/python3.7/site-packages/torch/lib/include -I/cluster/apps/python/3.7.1/x86_64/lib64/python3.7/site-packages/torch/lib/include/torch/csrc/api/include -I/cluster/apps/python/3.7.1/x86_64/lib64/python3.7/site-packages/torch/lib/include/TH -I/cluster/apps/python/3.7.1/x86_64/lib64/python3.7/site-packages/torch/lib/include/THC -I/cluster/apps/gcc-4.8.5/cuda-10.0.130-y7x264d5jchmrxfbsqt2ztwerqrrwzru/include -I/cluster/apps/python/3.7.1/x86_64/include/python3.7m -c sparseconvnet/SCN/cuda.cu -o build/temp.linux-x86_64-3.7/sparseconvnet/SCN/cuda.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --compiler-options '-fPIC' -std=c++11 -Xcompiler -fopenmp -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=SCN -D_GLIBCXX_USE_CXX11_ABI=0
/usr/bin/gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -ftree-vectorize -march=core-avx2 -mavx2 -ftree-vectorize -march=core-avx2 -mavx2 -fPIC -I/cluster/apps/python/3.7.1/x86_64/include -I/cluster/home/sfux/test/hanxue/SparseConvNet/sparseconvnet/SCN/ -I/cluster/apps/python/3.7.1/x86_64/lib64/python3.7/site-packages/torch/lib/include -I/cluster/apps/python/3.7.1/x86_64/lib64/python3.7/site-packages/torch/lib/include/torch/csrc/api/include -I/cluster/apps/python/3.7.1/x86_64/lib64/python3.7/site-packages/torch/lib/include/TH -I/cluster/apps/python/3.7.1/x86_64/lib64/python3.7/site-packages/torch/lib/include/THC -I/cluster/apps/gcc-4.8.5/cuda-10.0.130-y7x264d5jchmrxfbsqt2ztwerqrrwzru/include -I/cluster/apps/python/3.7.1/x86_64/include/python3.7m -c sparseconvnet/SCN/sparseconvnet_cuda.cpp -o build/temp.linux-x86_64-3.7/sparseconvnet/SCN/sparseconvnet_cuda.o -std=c++11 -fopenmp -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=SCN -D_GLIBCXX_USE_CXX11_ABI=0
/usr/bin/gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -ftree-vectorize -march=core-avx2 -mavx2 -ftree-vectorize -march=core-avx2 -mavx2 -fPIC -I/cluster/apps/python/3.7.1/x86_64/include -I/cluster/home/sfux/test/hanxue/SparseConvNet/sparseconvnet/SCN/ -I/cluster/apps/python/3.7.1/x86_64/lib64/python3.7/site-packages/torch/lib/include -I/cluster/apps/python/3.7.1/x86_64/lib64/python3.7/site-packages/torch/lib/include/torch/csrc/api/include -I/cluster/apps/python/3.7.1/x86_64/lib64/python3.7/site-packages/torch/lib/include/TH -I/cluster/apps/python/3.7.1/x86_64/lib64/python3.7/site-packages/torch/lib/include/THC -I/cluster/apps/gcc-4.8.5/cuda-10.0.130-y7x264d5jchmrxfbsqt2ztwerqrrwzru/include -I/cluster/apps/python/3.7.1/x86_64/include/python3.7m -c sparseconvnet/SCN/pybind.cpp -o build/temp.linux-x86_64-3.7/sparseconvnet/SCN/pybind.o -std=c++11 -fopenmp -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=SCN -D_GLIBCXX_USE_CXX11_ABI=0
creating build/lib.linux-x86_64-3.7
creating build/lib.linux-x86_64-3.7/sparseconvnet
/usr/bin/g++ -pthread -shared build/temp.linux-x86_64-3.7/sparseconvnet/SCN/cuda.o build/temp.linux-x86_64-3.7/sparseconvnet/SCN/sparseconvnet_cuda.o build/temp.linux-x86_64-3.7/sparseconvnet/SCN/pybind.o -L/cluster/apps/gcc-4.8.5/cuda-10.0.130-y7x264d5jchmrxfbsqt2ztwerqrrwzru/lib64 -L/cluster/apps/python/3.7.1/x86_64/lib64 -lcudart -lpython3.7m -o build/lib.linux-x86_64-3.7/sparseconvnet/SCN.cpython-37m-x86_64-linux-gnu.so
copying build/lib.linux-x86_64-3.7/sparseconvnet/SCN.cpython-37m-x86_64-linux-gnu.so -> sparseconvnet
Creating /cluster/home/sfux/.local/lib/python3.7/site-packages/sparseconvnet.egg-link (link to .)
sparseconvnet 0.2 is already the active version in easy-install.pth

Installed /cluster/home/sfux/test/hanxue/SparseConvNet
Processing dependencies for sparseconvnet==0.2
Finished processing dependencies for sparseconvnet==0.2
develop.sh: line 10: 27089 Segmentation fault      (core dumped) python examples/hello-world.py
[sfux@lo-s4-040 SparseConvNet]$

[hanxuel]

I meet the same problem when trying to install spaseconvnet on cluster of university and still get stuck here.

[btgraham]

I am using gcc 5.3.0 to build the extension to work with the conda version of PyTorch.