On CUDA, check that a team policy won't throw due to team_size too large exception

E3SM-Project / HOMMEXX

Clone of ACME for CMDV-SE project to convert HOMME to C++

11 stars 0 forks source link

On CUDA, check that a team policy won't throw due to team_size too large exception #293

Open bartgol opened 6 years ago

bartgol commented 6 years ago

Kokkos asks Cuda to compute the max number of blocks, which is then used to compute the max team_size. If the requested team_size is too large, it throws.

Currently, our simulations never crashed, as long as build type of kokkos and homme match (debug-debug or release-release). However, we should not rely on this being always the case. Hence, upon policy creation, check if policy numbers are ok; if not, keep reducing either vector_length or team_size (depending if threads are preferred) until they are.

ambrad commented 6 years ago

If we're going to handle this, then we might just call

Kokkos::Impl::CudaGetMaxBlockSize<Kokkos::Impl::ParallelFor<Functor> >
::cuda_get_max_block_size

(or ParallelReduce, etc) directly. Would have to think how to do this so lambdas work, too. Maybe we could have top-level dispatches that wrap the Kokkos ones.