search

kokkos / kokkos-kernels

Kokkos C++ Performance Portability Programming Ecosystem: Math Kernels - Provides BLAS, Sparse BLAS and Graph Kernels
Other
304 stars 96 forks source link

Compilation error in Test_Graph_coarsen.hpp: error: static assertion failed with "Incompatible View copy construction" #1536

Closed ndellingwood closed 2 years ago

ndellingwood commented 2 years ago

In cuda/11.2 builds (CuSparse and CuBlas OFF) I'm seeing these compilation errors with develop branches of kokkos and kokkos-kernels:

cd /ascldap/users/ndellin/trilinos/Trilinos/Build/ATDM-Weaver-Cuda11/kokkos-kernels/graph/unit_test && /home/projects/ppc64le-pwr9-nvidia/spack-installs/openmpi/4.1.1/gcc/8.3.1/base/elujn3i/bin/mpicxx -DGTEST_HAS_PTHREAD=0 -I/ascldap/users/ndellin/trilinos/Trilinos/Build/ATDM-Weaver-Cuda11 -I/ascldap/users/ndellin/trilinos/Trilinos/Build/ATDM-Weaver-Cuda11/kokkos-kernels/common -I/ascldap/users/ndellin/trilinos/Trilinos/kokkos-kernels/common -I/ascldap/users/ndellin/trilinos/Trilinos/Build/ATDM-Weaver-Cuda11/kokkos-kernels/batched -I/ascldap/users/ndellin/trilinos/Trilinos/kokkos-kernels/batched -I/ascldap/users/ndellin/trilinos/Trilinos/Build/ATDM-Weaver-Cuda11/kokkos-kernels/blas -I/ascldap/users/ndellin/trilinos/Trilinos/kokkos-kernels/blas -I/ascldap/users/ndellin/trilinos/Trilinos/Build/ATDM-Weaver-Cuda11/kokkos-kernels/graph -I/ascldap/users/ndellin/trilinos/Trilinos/kokkos-kernels/graph -I/ascldap/users/ndellin/trilinos/Trilinos/Build/ATDM-Weaver-Cuda11/kokkos-kernels -I/ascldap/users/ndellin/trilinos/Trilinos/kokkos-kernels/common/src -I/ascldap/users/ndellin/trilinos/Trilinos/kokkos-kernels/common/src/impl -I/ascldap/users/ndellin/trilinos/Trilinos/kokkos-kernels/common/src/tpls -I/ascldap/users/ndellin/trilinos/Trilinos/kokkos-kernels/common/unit_test -I/ascldap/users/ndellin/trilinos/Trilinos/kokkos-kernels/batched/src -I/ascldap/users/ndellin/trilinos/Trilinos/kokkos-kernels/batched/dense/src -I/ascldap/users/ndellin/trilinos/Trilinos/kokkos-kernels/batched/dense/impl -I/ascldap/users/ndellin/trilinos/Trilinos/kokkos-kernels/batched/dense/unit_test -I/ascldap/users/ndellin/trilinos/Trilinos/kokkos-kernels/batched/sparse/src -I/ascldap/users/ndellin/trilinos/Trilinos/kokkos-kernels/batched/sparse/impl -I/ascldap/users/ndellin/trilinos/Trilinos/kokkos-kernels/batched/sparse/unit_test -I/ascldap/users/ndellin/trilinos/Trilinos/kokkos-kernels/blas/src -I/ascldap/users/ndellin/trilinos/Trilinos/kokkos-kernels/blas/impl -I/ascldap/users/ndellin/trilinos/Trilinos/kokkos-kernels/blas/eti -I/ascldap/users/ndellin/trilinos/Trilinos/Build/ATDM-Weaver-Cuda11/kokkos-kernels/blas/eti -I/ascldap/users/ndellin/trilinos/Trilinos/kokkos-kernels/blas/tpls -I/ascldap/users/ndellin/trilinos/Trilinos/kokkos-kernels/graph/src -I/ascldap/users/ndellin/trilinos/Trilinos/kokkos-kernels/graph/impl -I/ascldap/users/ndellin/trilinos/Trilinos/kokkos-kernels/graph/eti -I/ascldap/users/ndellin/trilinos/Trilinos/Build/ATDM-Weaver-Cuda11/kokkos-kernels/graph/eti -I/ascldap/users/ndellin/trilinos/Trilinos/kokkos-kernels/sparse/src -I/ascldap/users/ndellin/trilinos/Trilinos/kokkos-kernels/sparse/impl -I/ascldap/users/ndellin/trilinos/Trilinos/kokkos-kernels/sparse/eti -I/ascldap/users/ndellin/trilinos/Trilinos/Build/ATDM-Weaver-Cuda11/kokkos-kernels/sparse/eti -I/ascldap/users/ndellin/trilinos/Trilinos/kokkos-kernels/sparse/tpls -I/ascldap/users/ndellin/trilinos/Trilinos/Build/ATDM-Weaver-Cuda11/test_common -I/ascldap/users/ndellin/trilinos/Trilinos/kokkos-kernels/test_common -I/ascldap/users/ndellin/trilinos/Trilinos/Build/ATDM-Weaver-Cuda11/kokkos-kernels/graph/unit_test -I/ascldap/users/ndellin/trilinos/Trilinos/kokkos-kernels/graph/unit_test -I/ascldap/users/ndellin/trilinos/Trilinos/Build/ATDM-Weaver-Cuda11/kokkos/core/src -I/ascldap/users/ndellin/trilinos/Trilinos/kokkos/core/src -I/ascldap/users/ndellin/trilinos/Trilinos/Build/ATDM-Weaver-Cuda11/kokkos -I/ascldap/users/ndellin/trilinos/Trilinos/kokkos/core/src/../../tpls/desul/include -I/ascldap/users/ndellin/trilinos/Trilinos/Build/ATDM-Weaver-Cuda11/kokkos/containers/src -I/ascldap/users/ndellin/trilinos/Trilinos/kokkos/containers/src -I/ascldap/users/ndellin/trilinos/Trilinos/Build/ATDM-Weaver-Cuda11/kokkos/algorithms/src -I/ascldap/users/ndellin/trilinos/Trilinos/kokkos/algorithms/src -I/ascldap/users/ndellin/trilinos/Trilinos/kokkos-kernels/tpls/gtest -isystem /ascldap/users/projects/ppc64le-pwr9-nvidia/spack-installs/cuda/11.2.2/gcc/8.3.1/base/uflzqtv/include -pedantic -Wall -Wno-long-long -Wwrite-strings  -mcpu=power9 -mtune=power9  -expt-extended-lambda -Wext-lambda-captures-this -arch=sm_70  -O3 -DNDEBUG -std=c++17 -MD -MT kokkos-kernels/graph/unit_test/CMakeFiles/KokkosKernels_graph_cuda.dir/backends/Test_Cuda_Graph.cpp.o -MF CMakeFiles/KokkosKernels_graph_cuda.dir/backends/Test_Cuda_Graph.cpp.o.d -o CMakeFiles/KokkosKernels_graph_cuda.dir/backends/Test_Cuda_Graph.cpp.o -c /ascldap/users/ndellin/trilinos/Trilinos/kokkos-kernels/graph/unit_test/backends/Test_Cuda_Graph.cpp
/ascldap/users/ndellin/trilinos/Trilinos/kokkos/core/src/Kokkos_View.hpp(1336): error: static assertion failed with "Incompatible View copy construction"
          detected during:
            instantiation of "Kokkos::View<DataType, Properties...>::View(const Kokkos::View<RT, RP...> &, std::enable_if_t<Kokkos::Impl::ViewMapping<Kokkos::View<DataType, Properties...>::traits, Kokkos::View<RT, RP...>::traits, Kokkos::ViewTraits<DataType, Properties...>::specialize>::is_assignable_data_type, void> *) [with DataType=int *, Properties=<std::conditional_t<true, Kokkos::Cuda::array_layout, Kokkos::Cuda::array_layout>, Kokkos::Device<Kokkos::HostSpace::execution_space, Kokkos::Cuda::memory_space>, std::conditional_t<false, void, Kokkos::Experimental::DefaultViewHooks>>, RT=int *, RP=<std::conditional_t<true, Kokkos::Cuda::array_layout, Kokkos::Cuda::array_layout>, Kokkos::HostSpace>]"
/ascldap/users/ndellin/trilinos/Trilinos/kokkos-kernels/graph/unit_test/Test_Graph_coarsen.hpp(192): here
            instantiation of "__nv_bool verify_aggregator(crsMat, crsMat) [with crsMat=KokkosSparse::CrsMatrix<default_scalar, int, Kokkos::Cuda, void, size_t>]"
/ascldap/users/ndellin/trilinos/Trilinos/kokkos-kernels/graph/unit_test/Test_Graph_coarsen.hpp(405): here
            instantiation of "void test_coarsen_random<scalar,lno_t,size_type,device>(lno_t, size_type, lno_t, lno_t) [with scalar=double, lno_t=int, size_type=size_t, device=Kokkos::Cuda]"
/ascldap/users/ndellin/trilinos/Trilinos/kokkos-kernels/graph/unit_test/Test_Graph_coarsen.hpp(470): here

/ascldap/users/ndellin/trilinos/Trilinos/kokkos/core/src/impl/Kokkos_ViewMapping.hpp(3571): error: static assertion failed with "View assignment must have compatible spaces"
          detected during:
            instantiation of "void Kokkos::Impl::ViewMapping<DstTraits, SrcTraits, std::enable_if_t<<expression>, void>>::assign(Kokkos::Impl::ViewMapping<DstTraits, SrcTraits, std::enable_if_t<<expression>, void>>::DstType &, const Kokkos::Impl::ViewMapping<DstTraits, SrcTraits, std::enable_if_t<<expression>, void>>::SrcType &, const Kokkos::Impl::ViewMapping<DstTraits, SrcTraits, std::enable_if_t<<expression>, void>>::TrackType &) [with DstTraits=Kokkos::ViewTraits<int *, std::conditional_t<true, Kokkos::Cuda::array_layout, Kokkos::Cuda::array_layout>, Kokkos::Device<Kokkos::HostSpace::execution_space, Kokkos::Cuda::memory_space>, std::conditional_t<false, void, Kokkos::Experimental::DefaultViewHooks>>, SrcTraits=Kokkos::ViewTraits<int *, std::conditional_t<true, Kokkos::Cuda::array_layout, Kokkos::Cuda::array_layout>, Kokkos::HostSpace>]"
/ascldap/users/ndellin/trilinos/Trilinos/kokkos/core/src/Kokkos_View.hpp(1338): here
            instantiation of "Kokkos::View<DataType, Properties...>::View(const Kokkos::View<RT, RP...> &, std::enable_if_t<Kokkos::Impl::ViewMapping<Kokkos::View<DataType, Properties...>::traits, Kokkos::View<RT, RP...>::traits, Kokkos::ViewTraits<DataType, Properties...>::specialize>::is_assignable_data_type, void> *) [with DataType=int *, Properties=<std::conditional_t<true, Kokkos::Cuda::array_layout, Kokkos::Cuda::array_layout>, Kokkos::Device<Kokkos::HostSpace::execution_space, Kokkos::Cuda::memory_space>, std::conditional_t<false, void, Kokkos::Experimental::DefaultViewHooks>>, RT=int *, RP=<std::conditional_t<true, Kokkos::Cuda::array_layout, Kokkos::Cuda::array_layout>, Kokkos::HostSpace>]"
/ascldap/users/ndellin/trilinos/Trilinos/kokkos-kernels/graph/unit_test/Test_Graph_coarsen.hpp(192): here
            instantiation of "__nv_bool verify_aggregator(crsMat, crsMat) [with crsMat=KokkosSparse::CrsMatrix<default_scalar, int, Kokkos::Cuda, void, size_t>]"
/ascldap/users/ndellin/trilinos/Trilinos/kokkos-kernels/graph/unit_test/Test_Graph_coarsen.hpp(405): here
            instantiation of "void test_coarsen_random<scalar,lno_t,size_type,device>(lno_t, size_type, lno_t, lno_t) [with scalar=double, lno_t=int, size_type=size_t, device=Kokkos::Cuda]"
ndellingwood commented 2 years ago

This is with this instantiation of the unit test: EXECUTE_TEST(double, int, size_t, TestExecSpace) (i.e. line 470 of kokkos-kernels/graph/unit_test/Test_Graph_coarsen.hpp)

ndellingwood commented 2 years ago

Replacing usage of HostMirror by auto to provide the type for host mirror Views resolved the compilation errors, I'm not sure if this change might be masking something though (i.e. maybe some missing ETI for size_t with CrsMatrix or Graph)? @lucbv @brian-kelley any thoughts? I'm posting my diff below and will put in a tentative PR to get it started testing in case it is an acceptable path to resolve this

diff --git a/graph/unit_test/Test_Graph_coarsen.hpp b/graph/unit_test/Test_Graph_coarsen.hpp
index af8ec55..77da10e 100644
--- a/graph/unit_test/Test_Graph_coarsen.hpp
+++ b/graph/unit_test/Test_Graph_coarsen.hpp
@@ -81,17 +81,17 @@ bool verify_coarsening(typename coarsener_t::coarse_level_triple fine_l,
       Kokkos::create_mirror_view_and_copy(Kokkos::HostSpace(), A.graph.row_map);
   auto c_rowmap = Kokkos::create_mirror_view_and_copy(Kokkos::HostSpace(),
                                                       coarse_A.graph.row_map);
-  typename c_entries_t::HostMirror f_entries =
+  auto f_entries =
       Kokkos::create_mirror_view_and_copy(Kokkos::HostSpace(), A.graph.entries);
-  typename c_entries_t::HostMirror vcmap = Kokkos::create_mirror_view_and_copy(
+  auto vcmap = Kokkos::create_mirror_view_and_copy(
       Kokkos::HostSpace(), coarse_l.interp_mtx.graph.entries);
-  typename svt::HostMirror few =
+  auto few =
       Kokkos::create_mirror_view_and_copy(Kokkos::HostSpace(), A.values);
-  typename svt::HostMirror cew =
+  auto cew =
       Kokkos::create_mirror_view_and_copy(Kokkos::HostSpace(), coarse_A.values);
-  typename entries_t::HostMirror fvw =
+  auto fvw =
       Kokkos::create_mirror_view_and_copy(Kokkos::HostSpace(), fine_l.vtx_wgts);
-  typename entries_t::HostMirror cvw = Kokkos::create_mirror_view_and_copy(
+  auto cvw = Kokkos::create_mirror_view_and_copy(
       Kokkos::HostSpace(), coarse_l.vtx_wgts);
   ordinal_t f_size = 0;
   ordinal_t c_size = 0;
@@ -143,7 +143,7 @@ bool verify_is_graph(crsMat A) {
   using edge_t      = typename rowmap_t::value_type;
   auto rowmap =
       Kokkos::create_mirror_view_and_copy(Kokkos::HostSpace(), A.graph.row_map);
-  typename c_entries_t::HostMirror entries =
+  auto entries =
       Kokkos::create_mirror_view_and_copy(Kokkos::HostSpace(), A.graph.entries);

   for (ordinal_t i = 0; i < A.numRows(); i++) {
@@ -187,7 +187,7 @@ bool verify_aggregator(crsMat A, crsMat agg) {
   if (A.numRows() < agg.numCols()) {
     return false;
   }
-  typename c_entries_t::HostMirror entries =
+  auto entries =
       Kokkos::create_mirror_view_and_copy(Kokkos::HostSpace(),
                                           agg.graph.entries);
ndellingwood commented 2 years ago

Potentially resolved by #1538

brian-kelley commented 2 years ago

@ndellingwood This is fine. Nothing in grpah is ETI'd except D1 coloring so it won't be a problem now. If we ETI coarsening later, we can just adjust the test then to build with ETI-only types.

brian-kelley commented 2 years ago

Actually even then this will be fine because these mirrors are only used within the test, not passed into kernels.