Trouble building galario for GPU use

[bjnorfolk]

Hi, I believe I've correctly installed all the required dependencies but I keep getting the error:

Scanning dependencies of target galario_single
[  4%] Building CXX object src/CMakeFiles/galario_single.dir/galario.cpp.o
/home/brodie/git_python/galario/src/galario.cpp: In function ‘void create_image_h(int, const dreal*, dreal, dreal, int, dreal, dreal, dcomplex*)’:
/home/brodie/git_python/galario/src/galario.cpp:1196:28: warning: ‘void* memset(void*, int, size_t)’ clearing an object of non-trivial type ‘dcomplex’ {aka ‘struct std::complex<float>’}; use assignment or value-initialization instead [-Wclass-memaccess]
 1196 |     memset(image, 0, nbytes);
      |                            ^
In file included from /home/brodie/git_python/galario/src/galario_defs.h:25,
                 from /home/brodie/git_python/galario/src/galario.h:22,
                 from /home/brodie/git_python/galario/src/galario.cpp:20:
/usr/include/c++/10/complex:1082:12: note: ‘dcomplex’ {aka ‘struct std::complex<float>’} declared here
 1082 |     struct complex<float>
      |            ^~~~~~~~~~~~~~
[  8%] Linking CXX shared library libgalario_single.so
[  8%] Built target galario_single
[ 13%] Building NVCC (Device) object src/CMakeFiles/galario_single_cuda.dir/galario_single_cuda_generated_cuda_lib.cu.o
nvcc fatal   : Value 'compute_30' is not defined for option 'gpu-architecture'
CMake Error at galario_single_cuda_generated_cuda_lib.cu.o.Release.cmake:220 (message):
  Error generating
  /home/brodie/git_python/galario/build/src/CMakeFiles/galario_single_cuda.dir//./galario_single_cuda_generated_cuda_lib.cu.o

make[2]: *** [src/CMakeFiles/galario_cuda.dir/build.make:84: src/CMakeFiles/galario_single_cuda.dir/galario_single_cuda_generated_cuda_lib.cu.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:227: src/CMakeFiles/galario_cuda.dir/all] Error 2
make: *** [Makefile:160: all] Error 2

When executing make && make install within the build folder. Any tips?

[bjnorfolk]

I should add, I'm on ubuntu 20.10 and have CUDA toolkit 11.2 installed.

[mtazzari]

Hi! Glad to see you are trying out the GPU version too! Thanks for reporting the issue. I was recently reported of some issues with the most recent CUDA version (v11). I built galario just fine with CUDA <=10 both on Linux and Mac OS. I haven't tried with CUDA 11 myself but judging from the error you get it is possible that in release 11 NVIDIA dropped compatibility with some older architectures. While I dig deeper to find a fix, I suggest you to try with CUDA 10. It should work fine and there'll be no change in performance vs CUDA 11. Let me know how it goes...

[bjnorfolk]

Hi Marco, thanks for the reply. I've attempted to troubleshoot with CUDA 10 extensively and I've run into a new issue:

CMake Error: The following variables are used in this project, but they are set to NOTFOUND.
Please set them or make sure they are set and tested correctly in the CMake files:
CUDA_cublas_LIBRARY (ADVANCED)
    linked by target "galario_cuda" in directory /home/brodie/git_python/galario/src
    linked by target "galario_single_cuda" in directory /home/brodie/git_python/galario/src

Now I assume this is a CUDA issue but did you have a work around? (I'm using cmake 3.20)

[bjnorfolk]

I did try the work-around cmake .. -DCUDA_cublas_LIBRARY=/usr/lib/x86_64-linux-gnu/libcublas.so.9.1 but that only resulted in the make error:

make[2]: *** No rule to make target '/usr/lib/x86_64-linux-gnu/libcublas.so.9.1', needed by 'src/libgalario_cuda.so'.  Stop.
make[1]: *** [CMakeFiles/Makefile2:205: src/CMakeFiles/galario_cuda.dir/all] Error 2
make: *** [Makefile:146: all] Error 2

[bjnorfolk]

Figured it out with cmake .. -DCUDA_cublas_LIBRARY=/usr/lib/x86_64-linux-gnu/libcublas.so and sudo ln -s /usr/local/cuda-10.0/targets/x86_64-linux/lib/libcublas.so /usr/lib/x86_64-linux-gnu/libcublas.so then make

[mtazzari]

Great you solved it. Did you try:

 cmake .. -DCUDA_cublas_LIBRARY=/usr/local/cuda-10.0/targets/x86_64-linux/lib/libcublas.so

[EDIT] without need for sudo ln -s ....

Just wondering why you need to ln -s to a second path instead of setting CUDA_cublas_LIBRARY to the actual path where you have libcublas.so. Thanks!

[bjnorfolk]

That does make a lot more sense, thanks Marco!