Cannot use MGARD with CUDA

[kmorel]

It does not seem to be possible to compile an MGARD application with nvcc. More specifically, if you have include compress.hpp, nvcc gives some strange compile errors. This is a problem since functions like mgard_cuda::compress are defined in compress.hpp.

I can replicate the problem with the following trivial source file:

#include <compress.hpp>

int main(int, char**)
{
  return 0;
}

When I attempt to compile this with nvcc, I get the following errors:

/ccs/home/kmorel/src/MGARD/include/utilities.tpp:248:29: error: '__T0' was not declared in this scope
     const CartesianProduct<T, N> &iterable,
                             ^~~~
/ccs/home/kmorel/src/MGARD/include/utilities.tpp:248:29: note: suggested alternative: '__y0'
     const CartesianProduct<T, N> &iterable,
                             ^~~~
                             __y0
/ccs/home/kmorel/src/MGARD/include/utilities.tpp:248:36: error: template argument 1 is invalid
     const CartesianProduct<T, N> &iterable,
                                    ^
/ccs/home/kmorel/src/MGARD/include/utilities.tpp:247:1: error: no declaration matches 'mgard::CartesianProduct<T, N>::iterator::iterator(const mgard::CartesianProduct<T, N>&, int)'
 CartesianProduct<T, N>::iterator::iterator(
 ^~~~~~~~~~~~~~~~~~~~~~~
/ccs/home/kmorel/src/MGARD/include/utilities.hpp:363:1: note: candidate is: 'mgard::CartesianProduct<T, N>::iterator::iterator(const mgard::CartesianProduct<T, N>&, std::array<decltype (declval<typename std::array<_Tp, _Nm>::const_reference>().begin()), N>)'
   iterator(const CartesianProduct &iterable,
 ^ ~~~~~~
/ccs/home/kmorel/src/MGARD/include/utilities.hpp:338:67: note: 'class mgard::CartesianProduct<T, N>::iterator' defined here
 template <typename T, std::size_t N> class CartesianProduct<T, N>::iterator {
                                                                   ^~~~~~~~

To make it easy to replicate, here is a tiny CMake project that demonstrates the problem: mgardcudatest.tar.gz.

Here is a capture of me replicating this issue on summit.

login1 0> cat mgardcudatest.cu
#include <compress.hpp>

int main(int, char**)
{
  return 0;
}

login1 0> cat CMakeLists.txt
cmake_minimum_required(VERSION 3.8)
project(mgardcudatest LANGUAGES CXX CUDA)

find_package(mgard REQUIRED)

add_executable(mgardcudatest mgardcudatest.cu)
target_link_libraries(mgardcudatest mgard::mgard)

login1 0> mkdir build
login1 0> cd build
login1 0> module purge
login1 0> module load cuda/11.4.2
login1 0> module load cmake
login1 0> cmake -Dmgard_DIR=$HOME/builds/MGARD -Dnvcomp_DIR=$HOME/builds/nvcomp ..
-- The CXX compiler identification is GNU 8.3.1
-- The CUDA compiler identification is NVIDIA 11.4.120
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Detecting CUDA compiler ABI info
-- Detecting CUDA compiler ABI info - done
-- Check for working CUDA compiler: /sw/summit/cuda/11.4.2/bin/nvcc - skipped
-- Detecting CUDA compile features
-- Detecting CUDA compile features - done
-- Found ZLIB: /usr/lib64/libz.so (found version "1.2.11") 
-- Found CUDAToolkit: /sw/summit/cuda/11.4.2/include (found version "11.4.120") 
-- Looking for C++ include pthread.h
-- Looking for C++ include pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE  
-- Found nvcomp: /ccs/home/kmorel/builds/nvcomp/nvcomp-config.cmake  
-- Found OpenMP_CXX: -fopenmp (found version "4.5") 
-- Found OpenMP: TRUE (found version "4.5")  
-- Found mgard: /ccs/home/kmorel/builds/MGARD/mgard-config.cmake (found version "1.0.0") 
-- Configuring done
CMake Warning (dev) in CMakeLists.txt:
  Policy CMP0104 is not set: CMAKE_CUDA_ARCHITECTURES now detected for NVCC,
  empty CUDA_ARCHITECTURES not allowed.  Run "cmake --help-policy CMP0104"
  for policy details.  Use the cmake_policy command to set the policy and
  suppress this warning.

  CUDA_ARCHITECTURES is empty for target "mgardcudatest".
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Warning (dev) in CMakeLists.txt:
  Policy CMP0104 is not set: CMAKE_CUDA_ARCHITECTURES now detected for NVCC,
  empty CUDA_ARCHITECTURES not allowed.  Run "cmake --help-policy CMP0104"
  for policy details.  Use the cmake_policy command to set the policy and
  suppress this warning.

  CUDA_ARCHITECTURES is empty for target "mgardcudatest".
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Generating done
-- Build files have been written to: /ccs/home/kmorel/tmp/build
login1 0> cmake --build .
[ 50%] Building CUDA object CMakeFiles/mgardcudatest.dir/mgardcudatest.cu.o
/ccs/home/kmorel/src/MGARD/include/utilities.tpp:248:29: error: '__T0' was not declared in this scope
     const CartesianProduct<T, N> &iterable,
                             ^~~~
/ccs/home/kmorel/src/MGARD/include/utilities.tpp:248:29: note: suggested alternative: '__y0'
     const CartesianProduct<T, N> &iterable,
                             ^~~~
                             __y0
/ccs/home/kmorel/src/MGARD/include/utilities.tpp:248:36: error: template argument 1 is invalid
     const CartesianProduct<T, N> &iterable,
                                    ^
/ccs/home/kmorel/src/MGARD/include/utilities.tpp:247:1: error: no declaration matches 'mgard::CartesianProduct<T, N>::iterator::iterator(const mgard::CartesianProduct<T, N>&, int)'
 CartesianProduct<T, N>::iterator::iterator(
 ^~~~~~~~~~~~~~~~~~~~~~~
/ccs/home/kmorel/src/MGARD/include/utilities.hpp:363:1: note: candidate is: 'mgard::CartesianProduct<T, N>::iterator::iterator(const mgard::CartesianProduct<T, N>&, std::array<decltype (declval<typename std::array<_Tp, _Nm>::const_reference>().begin()), N>)'
   iterator(const CartesianProduct &iterable,
 ^ ~~~~~~
/ccs/home/kmorel/src/MGARD/include/utilities.hpp:338:67: note: 'class mgard::CartesianProduct<T, N>::iterator' defined here
 template <typename T, std::size_t N> class CartesianProduct<T, N>::iterator {
                                                                   ^~~~~~~~
gmake[2]: *** [CMakeFiles/mgardcudatest.dir/build.make:76: CMakeFiles/mgardcudatest.dir/mgardcudatest.cu.o] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:83: CMakeFiles/mgardcudatest.dir/all] Error 2
gmake: *** [Makefile:91: all] Error 2

[JieyangChen7]

@kmorel This is a known issue caused by a bug in NVCC (bug report). Currently, we workaround this problem by having two headers: compress.hpp for user's program that is to be compiled with C/C++ compilers and compress_cuda.hpp for user's program that is to be compiled with NVCC compiler. Users can invoke the GPU compression using both methods. You can refer to the document for MGRAD-GPU for more details about headers and APIs in here.

[kmorel]

Thanks for the clarification and pointing me to the MGARD-GPU documentation.