Dispatcher-Single tests failed with OpenMPTarget

[chichunchen]

Below three tests weren't compiled with either Clang-15 or gcc-11.1.0.

        565 - test-workgroup-Dispatcher-Single-Direct-OpenMPTarget.exe (Not Run)
        566 - test-workgroup-Dispatcher-Single-IndirectFunction-OpenMPTarget.exe (Not Run)
        567 - test-workgroup-Dispatcher-Single-IndirectVirtual-OpenMPTarget.exe (Not Run)

All three tests can be compiled if removing the argument in makeDispatcher in include/RAJA/policy/openmp_target/WorkGroup/Dispatcher.hpp

/*!
* Populate and return a Dispatcher object that can be used in omp target regions
*/
template < typename T, typename Dispatcher_T >
inline const Dispatcher_T* get_Dispatcher(omp_target_work const&)
{
  static Dispatcher_T dispatcher{
        Dispatcher_T::template makeDispatcher<T>(
          [](auto&& factory) {             <- can compile the tests if remove this argument
            return omp_target::get_cached_value(
                std::forward<decltype(factory)>(factory));
          }) };
  return &dispatcher;
}

CMake configuration:

cmake -DENABLE_OPENMP=On \
    -DENABLE_TARGET_OPENMP=On \
    -DRAJA_ENABLE_TARGET_OPENMP=On \
    -DCMAKE_CXX_COMPILER=g++ \
    -DCMAKE_C_COMPILER=gcc \
    -DCMAKE_BUILD_TYPE=Release \
    -DCMAKE_CXX_STANDARD=14 \
    -DENABLE_TESTS=On \
    -DENABLE_EXAMPLES=Off  ../

Error message:

In file included from /ptmp/cchen/PE-42179/RAJA_master_2/build/test/unit/workgroup/test-workgroup-Dispatcher-Single-IndirectVirtual-OpenMPTarget.cpp:12:
In file included from /ptmp/cchen/PE-42179/RAJA_master_2/test/unit/workgroup/tests/test-workgroup-Dispatcher.hpp:15:
In file included from /ptmp/cchen/PE-42179/RAJA_master_2/test/include/RAJA_test-workgroup.hpp:11:
In file included from /ptmp/cchen/PE-42179/RAJA_master_2/test/include/RAJA_test-base.hpp:15:
In file included from /ptmp/cchen/PE-42179/RAJA_master_2/include/RAJA/RAJA.hpp:93:
In file included from /ptmp/cchen/PE-42179/RAJA_master_2/include/RAJA/policy/openmp_target.hpp:33:
In file included from /ptmp/cchen/PE-42179/RAJA_master_2/include/RAJA/policy/openmp_target/WorkGroup.hpp:21:
/ptmp/cchen/PE-42179/RAJA_master_2/include/RAJA/policy/openmp_target/WorkGroup/Dispatcher.hpp:70:9: error: no matching function for call to 'makeDispatcher'
        Dispatcher_T::template makeDispatcher<T>(
        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/ptmp/cchen/PE-42179/RAJA_master_2/test/unit/workgroup/tests/test-workgroup-Dispatcher.hpp:125:21: note: in instantiation of function template specialization 'RAJA::detail::get_Dispatcher<DispatcherTestCallable<int>, RAJA::detail::Dispatcher<camp::resources::Platform::omp
_target, RAJA::policy::workgroup::indirect_virtual_function_dispatch, void, int>>' requested here
      RAJA::detail::get_Dispatcher<TestCallable, Dispatcher_type>(ExecPolicy{});
                    ^
/ptmp/cchen/PE-42179/RAJA_master_2/test/unit/workgroup/tests/test-workgroup-Dispatcher.hpp:267:3: note: in instantiation of function template specialization 'testWorkGroupDispatcherSingle<RAJA::policy::omp::omp_target_work, detail::indirect_virtual_function_dispatch_typer
, int, camp::resources::Omp, forone_openmp_target>::operator()<>' requested here
  testWorkGroupDispatcherSingle< ExecPolicy, DispatchTyper, IndexType, ResourceType, ForOneType >{}(
  ^
/ptmp/cchen/PE-42179/RAJA_master_2/blt/thirdparty_builtin/googletest-master-2020-01-07/googletest/include/gtest/internal/gtest-internal.h:470:44: note: in instantiation of member function 'gtest_suite_WorkGroupBasicDispatcherSingleUnitTest_::BasicWorkGroupDispatcherSingle
<camp::list<RAJA::policy::omp::omp_target_work, detail::indirect_virtual_function_dispatch_typer, int, camp::list<>, camp::resources::Omp, forone_openmp_target>>::TestBody' requested here
  Test* CreateTest() override { return new TestClass; }
                                           ^
/ptmp/cchen/PE-42179/RAJA_master_2/blt/thirdparty_builtin/googletest-master-2020-01-07/googletest/include/gtest/internal/gtest-internal.h:728:13: note: in instantiation of member function 'testing::internal::TestFactoryImpl<gtest_suite_WorkGroupBasicDispatcherSingleUnitTe
st_::BasicWorkGroupDispatcherSingle<camp::list<RAJA::policy::omp::omp_target_work, detail::indirect_virtual_function_dispatch_typer, int, camp::list<>, camp::resources::Omp, forone_openmp_target>>>::CreateTest' requested here
        new TestFactoryImpl<TestClass>);
            ^
/ptmp/cchen/PE-42179/RAJA_master_2/blt/thirdparty_builtin/googletest-master-2020-01-07/googletest/include/gtest/internal/gtest-internal.h:787:50: note: in instantiation of member function 'testing::internal::TypeParameterizedTest<WorkGroupBasicDispatcherSingleUnitTest, te
sting::internal::TemplateSel<gtest_suite_WorkGroupBasicDispatcherSingleUnitTest_::BasicWorkGroupDispatcherSingle>, testing::internal::Types<camp::list<RAJA::policy::omp::omp_target_work, detail::indirect_virtual_function_dispatch_typer, int, camp::list<>, camp::resources:
:Omp, forone_openmp_target>, camp::list<RAJA::policy::omp::omp_target_work, detail::indirect_virtual_function_dispatch_typer, int, camp::list<int *>, camp::resources::Omp, forone_openmp_target>, camp::list<RAJA::policy::omp::omp_target_work, detail::indirect_virtual_funct
ion_dispatch_typer, int, camp::list<int, int *>, camp::resources::Omp, forone_openmp_target>, camp::list<RAJA::policy::omp::omp_target_work, detail::indirect_virtual_function_dispatch_typer, long, camp::list<>, camp::resources::Omp, forone_openmp_target>, camp::list<RAJA:
:policy::omp::omp_target_work, detail::indirect_virtual_function_dispatch_typer, long, camp::list<int *>, camp::resources::Omp, forone_openmp_target>, camp::list<RAJA::policy::omp::omp_target_work, detail::indirect_virtual_function_dispatch_typer, long, camp::list<int, in
t *>, camp::resources::Omp, forone_openmp_target>, camp::list<RAJA::policy::omp::omp_target_work, detail::indirect_virtual_function_dispatch_typer, long, camp::list<>, camp::resources::Omp, forone_openmp_target>, camp::list<RAJA::policy::omp::omp_target_work, detail::indi
rect_virtual_function_dispatch_typer, long, camp::list<int *>, camp::resources::Omp, forone_openmp_target>, camp::list<RAJA::policy::omp::omp_target_work, detail::indirect_virtual_function_dispatch_typer, long, camp::list<int, int *>, camp::resources::Omp, forone_openmp_t
arget>>>::Register' requested here
    TypeParameterizedTest<Fixture, Head, Types>::Register(
                                                 ^
/ptmp/cchen/PE-42179/RAJA_master_2/build/test/unit/workgroup/test-workgroup-Dispatcher-Single-IndirectVirtual-OpenMPTarget.cpp:25:1: note: in instantiation of member function 'testing::internal::TypeParameterizedTestSuite<WorkGroupBasicDispatcherSingleUnitTest, testing::i
nternal::Templates<gtest_suite_WorkGroupBasicDispatcherSingleUnitTest_::BasicWorkGroupDispatcherSingle>, testing::internal::Types<camp::list<RAJA::policy::omp::omp_target_work, detail::indirect_virtual_function_dispatch_typer, int, camp::list<>, camp::resources::Omp, foro
ne_openmp_target>, camp::list<RAJA::policy::omp::omp_target_work, detail::indirect_virtual_function_dispatch_typer, int, camp::list<int *>, camp::resources::Omp, forone_openmp_target>, camp::list<RAJA::policy::omp::omp_target_work, detail::indirect_virtual_function_dispat
ch_typer, int, camp::list<int, int *>, camp::resources::Omp, forone_openmp_target>, camp::list<RAJA::policy::omp::omp_target_work, detail::indirect_virtual_function_dispatch_typer, long, camp::list<>, camp::resources::Omp, forone_openmp_target>, camp::list<RAJA::policy::o
mp::omp_target_work, detail::indirect_virtual_function_dispatch_typer, long, camp::list<int *>, camp::resources::Omp, forone_openmp_target>, camp::list<RAJA::policy::omp::omp_target_work, detail::indirect_virtual_function_dispatch_typer, long, camp::list<int, int *>, camp
::resources::Omp, forone_openmp_target>, camp::list<RAJA::policy::omp::omp_target_work, detail::indirect_virtual_function_dispatch_typer, long, camp::list<>, camp::resources::Omp, forone_openmp_target>, camp::list<RAJA::policy::omp::omp_target_work, detail::indirect_virtu
al_function_dispatch_typer, long, camp::list<int *>, camp::resources::Omp, forone_openmp_target>, camp::list<RAJA::policy::omp::omp_target_work, detail::indirect_virtual_function_dispatch_typer, long, camp::list<int, int *>, camp::resources::Omp, forone_openmp_target>>>::
Register' requested here
INSTANTIATE_TYPED_TEST_SUITE_P(OpenMPTargetBasicTest,
^
/ptmp/cchen/PE-42179/RAJA_master_2/blt/thirdparty_builtin/googletest-master-2020-01-07/googletest/include/gtest/gtest-typed-test.h:317:11: note: expanded from macro 'INSTANTIATE_TYPED_TEST_SUITE_P'
          Register(GTEST_STRINGIFY_(Prefix),                                \
          ^
/ptmp/cchen/PE-42179/RAJA_master_2/include/RAJA/pattern/WorkGroup/Dispatcher.hpp:359:28: note: candidate template ignored: requirement '!true' was not satisfied [with T = DispatcherTestCallable<int>, CreateOnDevice = (lambda at /ptmp/cchen/PE-42179/RAJA_master_2/include/R
AJA/policy/openmp_target/WorkGroup/Dispatcher.hpp:71:11), uhi = true]
  static inline Dispatcher makeDispatcher(CreateOnDevice&& createOnDevice) {
                           ^
/ptmp/cchen/PE-42179/RAJA_master_2/include/RAJA/pattern/WorkGroup/Dispatcher.hpp:338:28: note: candidate function template not viable: requires 0 arguments, but 1 was provided
  static inline Dispatcher makeDispatcher() {
                           ^
6 errors generated.

[chichunchen]

        Dispatcher_T::template makeDispatcher<T>(
          [](auto&& factory) {   
            return omp_target::get_cached_value(
                std::forward<decltype(factory)>(factory));
          }) };

I can see that we'd like to choose the device version of makeDispatcher in "include/RAJA/pattern/WorkGroup/Dispatcher.hpp" but probably need to also provide the type of the lambda for makeDispatcher? I'm not familiar with C++ template metaprogramming so don't know if it's a compiler feature or something. I also tried compiling with C++17 but still got the same compile error.

[rhornung67]

@chichunchen do you need to use the OpenMP target offload back-end for RAJA? We don't consider it production ready, rather something we try to maintain it to test compilers and we don't spend much time on it. Depending on what you are trying to do and what hardware you are targeting, you will have better luck with performance, compiler and RAJA support with the CUDA or HIP back-ends. The RAJA SYCL back-end is in development, but a large fraction of features are supported already.

[MrBurmark]

Hmm, it looks like its trying to call the host version of makeDispatcher but it sohuld be calling the device version. The fix may be as simple as adding Platform::omp_target in dispatcher_use_host_invoke.