search

intel / oneapi-containers

BSD 3-Clause "New" or "Revised" License
111 stars 45 forks source link

Link error: `llvm-foreach: No such file or directory` #23

Closed pgrete closed 7 months ago

pgrete commented 2 years ago

I'm having troubles compiling (linking) code within the container seeing the following error:

[ 44%] Linking CXX executable KokkosExample_query_device
cd /build-test/example/query_device && /usr/local/bin/cmake -E cmake_link_script CMakeFiles/KokkosExample_query_device.dir/link.txt --verbose=1
/opt/intel/oneapi/compiler/2022.0.1/linux/bin/dpcpp -O2 -g -DNDEBUG -DKOKKOS_DEPENDENCE -fsycl -fno-sycl-id-queries-fit-in-int -fsycl-dead-args-optimization -fsycl-targets=spir64_gen -Xsycl-target-backend "-device gen12lp" CMakeFiles/KokkosExample_query_device.dir/query_device.cpp.o -o KokkosExample_query_device  ../../containers/src/libkokkoscontainers.a ../../core/src/libkokkoscore.a /usr/lib/x86_64-linux-gnu/libdl.so 
llvm-foreach: No such file or directory
dpcpp: error: gen compiler command failed with exit code 1 (use -v to see invocation)
example/query_device/CMakeFiles/KokkosExample_query_device.dir/build.make:99: recipe for target 'example/query_device/KokkosExample_query_device' failed
make[2]: *** [example/query_device/KokkosExample_query_device] Error 1
make[2]: Leaving directory '/build-test'

Steps to reproduce:

$ docker run -ti --device=/dev/dri intel/oneapi-basekit /bin/bash
$ git clone https://github.com/kokkos/kokkos.git -b develop
$ cmake -Bbuild-test -DKokkos_ARCH_INTEL_GEN12LP=ON -DKokkos_ENABLE_SYCL=ON -DCMAKE_CXX_STANDARD=17 -DCMAKE_CXX_COMPILER=dpcpp -DKokkos_ENABLE_EXAMPLES=ON kokkos
$ cmake --build build-test

I also tried adding llvm-foreach to the PATH as suggested by /opt/intel/oneapi/compiler/2022.0.1/linux/bin-llvm/README.txt but it didn't fix the issue.

Mentioning https://github.com/kokkos/kokkos/issues/4679 and https://github.com/intel/llvm/issues/2583 for reference.

luisfponce commented 2 years ago

Hello @pgrete,

I have a couple of questions before going deeper in this issue.

Finally, i tried to reproduce what you observed, I did not get the same output as you got

Commands:

EGPATH=/opt/intel/oneapi/vtune/2022.0.0/bin64:/opt/intel/oneapi/vpl/2022.0.0/bin:/opt/intel/oneapi/mpi/2021.5.0//libfabric/bin:/opt/intel/oneapi/mpi/2021.5.0//bin:/opt/intel/oneapi/mkl/2022.0.1/bin/intel64:/opt/intel/oneapi/intelpython/latest/bin:/opt/intel/oneapi/intelpython/latest/condabin:/opt/intel/oneapi/dev-utilities/2021.5.1/bin:/opt/intel/oneapi/debugger/2021.5.0/gdb/intel64/bin:/opt/intel/oneapi/compiler/2022.0.1/linux/lib/oclfpga/bin:/opt/intel/oneapi/compiler/2022.0.1/linux/bin/intel64:/opt/intel/oneapi/compiler/2022.0.1/linux/bin:/opt/intel/oneapi/advisor/2022.0.0/bin64:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/intel/oneapi/compiler/2022.0.1/linux/bin-llvm/llvm-foreach

docker run -ti --device=/dev/dri --env PATH=$EGPATH intel/oneapi-basekit:2021.4-devel-ubuntu18.04 /bin/bash
apt update && apt install -y git
git clone https://github.com/kokkos/kokkos.git -b develop
cmake -Bbuild-test -DKokkos_ARCH_INTEL_GEN12LP=ON -DKokkos_ENABLE_SYCL=ON -DCMAKE_CXX_STANDARD=17 -DCMAKE_CXX_COMPILER=dpcpp -DKokkos_ENABLE_EXAMPLES=ON kokkos
cmake --build build-test --verbose

output:

/usr/local/bin/cmake -S/kokkos -B/build-test --check-build-system CMakeFiles/Makefile.cmake 0
/usr/local/bin/cmake -E cmake_progress_start /build-test/CMakeFiles /build-test//CMakeFiles/progress.marks
/usr/bin/make  -f CMakeFiles/Makefile2 all
make[1]: Entering directory '/build-test'
/usr/bin/make  -f core/src/CMakeFiles/kokkoscore.dir/build.make core/src/CMakeFiles/kokkoscore.dir/depend
make[2]: Entering directory '/build-test'
cd /build-test && /usr/local/bin/cmake -E cmake_depends "Unix Makefiles" /kokkos /kokkos/core/src /build-test /build-test/core/src /build-test/core/src/CMakeFiles/kokkoscore.dir/DependInfo.cmake --color=
make[2]: Leaving directory '/build-test'
/usr/bin/make  -f core/src/CMakeFiles/kokkoscore.dir/build.make core/src/CMakeFiles/kokkoscore.dir/build
make[2]: Entering directory '/build-test'
[  1%] Building CXX object core/src/CMakeFiles/kokkoscore.dir/impl/Kokkos_CPUDiscovery.cpp.o
cd /build-test/core/src && /opt/intel/oneapi/compiler/2022.0.1/linux/bin/dpcpp -DKOKKOS_DEPENDENCE -I/build-test/core/src -I/kokkos/core/src -I/build-test -O2 -g -DNDEBUG -fsycl -fno-sycl-id-queries-fit-in-int -fsycl-unnamed-lambda -fsycl-targets=spir64_gen-unknown-unknown-sycldevice -Xsycl-target-backend "-device gen12lp" -MD -MT core/src/CMakeFiles/kokkoscore.dir/impl/Kokkos_CPUDiscovery.cpp.o -MF CMakeFiles/kokkoscore.dir/impl/Kokkos_CPUDiscovery.cpp.o.d -o CMakeFiles/kokkoscore.dir/impl/Kokkos_CPUDiscovery.cpp.o -c /kokkos/core/src/impl/Kokkos_CPUDiscovery.cpp
dpcpp: warning: argument 'spir64_gen-unknown-unknown-sycldevice' is deprecated, use 'spir64_gen' instead [-Wdeprecated]
[  3%] Building CXX object core/src/CMakeFiles/kokkoscore.dir/impl/Kokkos_Core.cpp.o
cd /build-test/core/src && /opt/intel/oneapi/compiler/2022.0.1/linux/bin/dpcpp -DKOKKOS_DEPENDENCE -I/build-test/core/src -I/kokkos/core/src -I/build-test -O2 -g -DNDEBUG -fsycl -fno-sycl-id-queries-fit-in-int -fsycl-unnamed-lambda -fsycl-targets=spir64_gen-unknown-unknown-sycldevice -Xsycl-target-backend "-device gen12lp" -MD -MT core/src/CMakeFiles/kokkoscore.dir/impl/Kokkos_Core.cpp.o -MF CMakeFiles/kokkoscore.dir/impl/Kokkos_Core.cpp.o.d -o CMakeFiles/kokkoscore.dir/impl/Kokkos_Core.cpp.o -c /kokkos/core/src/impl/Kokkos_Core.cpp
dpcpp: warning: argument 'spir64_gen-unknown-unknown-sycldevice' is deprecated, use 'spir64_gen' instead [-Wdeprecated]
In file included from /kokkos/core/src/impl/Kokkos_Core.cpp:45:
In file included from /kokkos/core/src/Kokkos_Core.hpp:51:
In file included from /kokkos/core/src/Kokkos_Core_fwd.hpp:53:
In file included from /kokkos/core/src/impl/Kokkos_Error.hpp:58:
/kokkos/core/src/SYCL/Kokkos_SYCL_Abort.hpp:55:3: error: no member named 'ONEAPI' in namespace 'sycl'
  KOKKOS_IMPL_DO_NOT_USE_PRINTF("Aborting with message %s.\n", msg);
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/kokkos/core/src/setup/Kokkos_Setup_SYCL.hpp:68:11: note: expanded from macro 'KOKKOS_IMPL_DO_NOT_USE_PRINTF'
    sycl::ONEAPI::experimental::printf(fmt, ##__VA_ARGS__);       \
    ~~~~~~^
In file included from /kokkos/core/src/impl/Kokkos_Core.cpp:45:
In file included from /kokkos/core/src/Kokkos_Core.hpp:56:
In file included from /build-test/KokkosCore_Config_DeclareBackend.hpp:47:
In file included from /kokkos/core/src/decl/Kokkos_Declare_SERIAL.hpp:49:
In file included from /kokkos/core/src/Kokkos_Serial.hpp:59:
In file included from /kokkos/core/src/Kokkos_Parallel.hpp:54:
In file included from /kokkos/core/src/Kokkos_View.hpp:500:
In file included from /kokkos/core/src/impl/Kokkos_ViewMapping.hpp:60:
In file included from /kokkos/core/src/impl/Kokkos_Atomic_View.hpp:48:
In file included from /kokkos/core/src/Kokkos_Atomic.hpp:77:
In file included from /kokkos/core/src/Kokkos_Atomics_Desul_Wrapper.hpp:6:
In file included from /kokkos/core/src/desul/atomics.hpp:14:
In file included from /kokkos/core/src/desul/atomics/Atomic_Ref.hpp:17:
In file included from /kokkos/core/src/desul/atomics/Generic.hpp:16:
In file included from /kokkos/core/src/desul/atomics/Compare_Exchange.hpp:33:
In file included from /kokkos/core/src/desul/atomics/Compare_Exchange_SYCL.hpp:12:
/kokkos/core/src/desul/atomics/SYCLConversions.hpp:21:20: error: no member named 'ONEAPI' in namespace 'sycl'
  static constexpr DESUL_SYCL_NAMESPACE::memory_order value = DESUL_SYCL_NAMESPACE::memory_order::seq_cst;
                   ^~~~~~~~~~~~~~~~~~~~
/kokkos/core/src/desul/atomics/Macros.hpp:38:36: note: expanded from macro 'DESUL_SYCL_NAMESPACE'
#define DESUL_SYCL_NAMESPACE sycl::ONEAPI
                             ~~~~~~^
In file included from /kokkos/core/src/impl/Kokkos_Core.cpp:45:
In file included from /kokkos/core/src/Kokkos_Core.hpp:56:
In file included from /build-test/KokkosCore_Config_DeclareBackend.hpp:47:
In file included from /kokkos/core/src/decl/Kokkos_Declare_SERIAL.hpp:49:
In file included from /kokkos/core/src/Kokkos_Serial.hpp:59:
In file included from /kokkos/core/src/Kokkos_Parallel.hpp:54:
In file included from /kokkos/core/src/Kokkos_View.hpp:500:
In file included from /kokkos/core/src/impl/Kokkos_ViewMapping.hpp:60:
In file included from /kokkos/core/src/impl/Kokkos_Atomic_View.hpp:48:
In file included from /kokkos/core/src/Kokkos_Atomic.hpp:77:
In file included from /kokkos/core/src/Kokkos_Atomics_Desul_Wrapper.hpp:6:
In file included from /kokkos/core/src/desul/atomics.hpp:14:
In file included from /kokkos/core/src/desul/atomics/Atomic_Ref.hpp:17:
In file included from /kokkos/core/src/desul/atomics/Generic.hpp:16:
In file included from /kokkos/core/src/desul/atomics/Compare_Exchange.hpp:33:
In file included from /kokkos/core/src/desul/atomics/Compare_Exchange_SYCL.hpp:12:
/kokkos/core/src/desul/atomics/SYCLConversions.hpp:21:63: error: no member named 'ONEAPI' in namespace 'sycl'
  static constexpr DESUL_SYCL_NAMESPACE::memory_order value = DESUL_SYCL_NAMESPACE::memory_order::seq_cst;
                                                              ^~~~~~~~~~~~~~~~~~~~
/kokkos/core/src/desul/atomics/Macros.hpp:38:36: note: expanded from macro 'DESUL_SYCL_NAMESPACE'
#define DESUL_SYCL_NAMESPACE sycl::ONEAPI
                             ~~~~~~^
In file included from /kokkos/core/src/impl/Kokkos_Core.cpp:45:
In file included from /kokkos/core/src/Kokkos_Core.hpp:56:
In file included from /build-test/KokkosCore_Config_DeclareBackend.hpp:47:
In file included from /kokkos/core/src/decl/Kokkos_Declare_SERIAL.hpp:49:
In file included from /kokkos/core/src/Kokkos_Serial.hpp:59:
In file included from /kokkos/core/src/Kokkos_Parallel.hpp:54:
In file included from /kokkos/core/src/Kokkos_View.hpp:500:
In file included from /kokkos/core/src/impl/Kokkos_ViewMapping.hpp:60:
In file included from /kokkos/core/src/impl/Kokkos_Atomic_View.hpp:48:
In file included from /kokkos/core/src/Kokkos_Atomic.hpp:77:
In file included from /kokkos/core/src/Kokkos_Atomics_Desul_Wrapper.hpp:6:
In file included from /kokkos/core/src/desul/atomics.hpp:14:
In file included from /kokkos/core/src/desul/atomics/Atomic_Ref.hpp:17:
In file included from /kokkos/core/src/desul/atomics/Generic.hpp:16:
In file included from /kokkos/core/src/desul/atomics/Compare_Exchange.hpp:33:
In file included from /kokkos/core/src/desul/atomics/Compare_Exchange_SYCL.hpp:12:
/kokkos/core/src/desul/atomics/SYCLConversions.hpp:25:20: error: no member named 'ONEAPI' in namespace 'sycl'
  static constexpr DESUL_SYCL_NAMESPACE::memory_order value = DESUL_SYCL_NAMESPACE::memory_order::acquire;
                   ^~~~~~~~~~~~~~~~~~~~
/kokkos/core/src/desul/atomics/Macros.hpp:38:36: note: expanded from macro 'DESUL_SYCL_NAMESPACE'
#define DESUL_SYCL_NAMESPACE sycl::ONEAPI
                             ~~~~~~^
In file included from /kokkos/core/src/impl/Kokkos_Core.cpp:45:
In file included from /kokkos/core/src/Kokkos_Core.hpp:56:
In file included from /build-test/KokkosCore_Config_DeclareBackend.hpp:47:
In file included from /kokkos/core/src/decl/Kokkos_Declare_SERIAL.hpp:49:
In file included from /kokkos/core/src/Kokkos_Serial.hpp:59:
In file included from /kokkos/core/src/Kokkos_Parallel.hpp:54:
In file included from /kokkos/core/src/Kokkos_View.hpp:500:
In file included from /kokkos/core/src/impl/Kokkos_ViewMapping.hpp:60:
In file included from /kokkos/core/src/impl/Kokkos_Atomic_View.hpp:48:
In file included from /kokkos/core/src/Kokkos_Atomic.hpp:77:
In file included from /kokkos/core/src/Kokkos_Atomics_Desul_Wrapper.hpp:6:
In file included from /kokkos/core/src/desul/atomics.hpp:14:
In file included from /kokkos/core/src/desul/atomics/Atomic_Ref.hpp:17:
In file included from /kokkos/core/src/desul/atomics/Generic.hpp:16:
In file included from /kokkos/core/src/desul/atomics/Compare_Exchange.hpp:33:
In file included from /kokkos/core/src/desul/atomics/Compare_Exchange_SYCL.hpp:12:
/kokkos/core/src/desul/atomics/SYCLConversions.hpp:25:63: error: no member named 'ONEAPI' in namespace 'sycl'
  static constexpr DESUL_SYCL_NAMESPACE::memory_order value = DESUL_SYCL_NAMESPACE::memory_order::acquire;
                                                              ^~~~~~~~~~~~~~~~~~~~
/kokkos/core/src/desul/atomics/Macros.hpp:38:36: note: expanded from macro 'DESUL_SYCL_NAMESPACE'
#define DESUL_SYCL_NAMESPACE sycl::ONEAPI
                             ~~~~~~^
In file included from /kokkos/core/src/impl/Kokkos_Core.cpp:45:
In file included from /kokkos/core/src/Kokkos_Core.hpp:56:
In file included from /build-test/KokkosCore_Config_DeclareBackend.hpp:47:
In file included from /kokkos/core/src/decl/Kokkos_Declare_SERIAL.hpp:49:
In file included from /kokkos/core/src/Kokkos_Serial.hpp:59:
In file included from /kokkos/core/src/Kokkos_Parallel.hpp:54:
In file included from /kokkos/core/src/Kokkos_View.hpp:500:
In file included from /kokkos/core/src/impl/Kokkos_ViewMapping.hpp:60:
In file included from /kokkos/core/src/impl/Kokkos_Atomic_View.hpp:48:
In file included from /kokkos/core/src/Kokkos_Atomic.hpp:77:
In file included from /kokkos/core/src/Kokkos_Atomics_Desul_Wrapper.hpp:6:
In file included from /kokkos/core/src/desul/atomics.hpp:14:
In file included from /kokkos/core/src/desul/atomics/Atomic_Ref.hpp:17:
In file included from /kokkos/core/src/desul/atomics/Generic.hpp:16:
In file included from /kokkos/core/src/desul/atomics/Compare_Exchange.hpp:33:
In file included from /kokkos/core/src/desul/atomics/Compare_Exchange_SYCL.hpp:12:
/kokkos/core/src/desul/atomics/SYCLConversions.hpp:29:20: error: no member named 'ONEAPI' in namespace 'sycl'
  static constexpr DESUL_SYCL_NAMESPACE::memory_order value = DESUL_SYCL_NAMESPACE::memory_order::release;
                   ^~~~~~~~~~~~~~~~~~~~
/kokkos/core/src/desul/atomics/Macros.hpp:38:36: note: expanded from macro 'DESUL_SYCL_NAMESPACE'
#define DESUL_SYCL_NAMESPACE sycl::ONEAPI
                             ~~~~~~^
In file included from /kokkos/core/src/impl/Kokkos_Core.cpp:45:
In file included from /kokkos/core/src/Kokkos_Core.hpp:56:
In file included from /build-test/KokkosCore_Config_DeclareBackend.hpp:47:
In file included from /kokkos/core/src/decl/Kokkos_Declare_SERIAL.hpp:49:
In file included from /kokkos/core/src/Kokkos_Serial.hpp:59:
In file included from /kokkos/core/src/Kokkos_Parallel.hpp:54:
In file included from /kokkos/core/src/Kokkos_View.hpp:500:
In file included from /kokkos/core/src/impl/Kokkos_ViewMapping.hpp:60:
In file included from /kokkos/core/src/impl/Kokkos_Atomic_View.hpp:48:
In file included from /kokkos/core/src/Kokkos_Atomic.hpp:77:
In file included from /kokkos/core/src/Kokkos_Atomics_Desul_Wrapper.hpp:6:
In file included from /kokkos/core/src/desul/atomics.hpp:14:
In file included from /kokkos/core/src/desul/atomics/Atomic_Ref.hpp:17:
In file included from /kokkos/core/src/desul/atomics/Generic.hpp:16:
In file included from /kokkos/core/src/desul/atomics/Compare_Exchange.hpp:33:
In file included from /kokkos/core/src/desul/atomics/Compare_Exchange_SYCL.hpp:12:
/kokkos/core/src/desul/atomics/SYCLConversions.hpp:29:63: error: no member named 'ONEAPI' in namespace 'sycl'
  static constexpr DESUL_SYCL_NAMESPACE::memory_order value = DESUL_SYCL_NAMESPACE::memory_order::release;
                                                              ^~~~~~~~~~~~~~~~~~~~
/kokkos/core/src/desul/atomics/Macros.hpp:38:36: note: expanded from macro 'DESUL_SYCL_NAMESPACE'
#define DESUL_SYCL_NAMESPACE sycl::ONEAPI
                             ~~~~~~^
In file included from /kokkos/core/src/impl/Kokkos_Core.cpp:45:
In file included from /kokkos/core/src/Kokkos_Core.hpp:56:
In file included from /build-test/KokkosCore_Config_DeclareBackend.hpp:47:
In file included from /kokkos/core/src/decl/Kokkos_Declare_SERIAL.hpp:49:
In file included from /kokkos/core/src/Kokkos_Serial.hpp:59:
In file included from /kokkos/core/src/Kokkos_Parallel.hpp:54:
In file included from /kokkos/core/src/Kokkos_View.hpp:500:
In file included from /kokkos/core/src/impl/Kokkos_ViewMapping.hpp:60:
In file included from /kokkos/core/src/impl/Kokkos_Atomic_View.hpp:48:
In file included from /kokkos/core/src/Kokkos_Atomic.hpp:77:
In file included from /kokkos/core/src/Kokkos_Atomics_Desul_Wrapper.hpp:6:
In file included from /kokkos/core/src/desul/atomics.hpp:14:
In file included from /kokkos/core/src/desul/atomics/Atomic_Ref.hpp:17:
In file included from /kokkos/core/src/desul/atomics/Generic.hpp:16:
In file included from /kokkos/core/src/desul/atomics/Compare_Exchange.hpp:33:
In file included from /kokkos/core/src/desul/atomics/Compare_Exchange_SYCL.hpp:12:
/kokkos/core/src/desul/atomics/SYCLConversions.hpp:33:20: error: no member named 'ONEAPI' in namespace 'sycl'
  static constexpr DESUL_SYCL_NAMESPACE::memory_order value = DESUL_SYCL_NAMESPACE::memory_order::acq_rel;
                   ^~~~~~~~~~~~~~~~~~~~
/kokkos/core/src/desul/atomics/Macros.hpp:38:36: note: expanded from macro 'DESUL_SYCL_NAMESPACE'
#define DESUL_SYCL_NAMESPACE sycl::ONEAPI
                             ~~~~~~^
In file included from /kokkos/core/src/impl/Kokkos_Core.cpp:45:
In file included from /kokkos/core/src/Kokkos_Core.hpp:56:
In file included from /build-test/KokkosCore_Config_DeclareBackend.hpp:47:
In file included from /kokkos/core/src/decl/Kokkos_Declare_SERIAL.hpp:49:
In file included from /kokkos/core/src/Kokkos_Serial.hpp:59:
In file included from /kokkos/core/src/Kokkos_Parallel.hpp:54:
In file included from /kokkos/core/src/Kokkos_View.hpp:500:
In file included from /kokkos/core/src/impl/Kokkos_ViewMapping.hpp:60:
In file included from /kokkos/core/src/impl/Kokkos_Atomic_View.hpp:48:
In file included from /kokkos/core/src/Kokkos_Atomic.hpp:77:
In file included from /kokkos/core/src/Kokkos_Atomics_Desul_Wrapper.hpp:6:
In file included from /kokkos/core/src/desul/atomics.hpp:14:
In file included from /kokkos/core/src/desul/atomics/Atomic_Ref.hpp:17:
In file included from /kokkos/core/src/desul/atomics/Generic.hpp:16:
In file included from /kokkos/core/src/desul/atomics/Compare_Exchange.hpp:33:
In file included from /kokkos/core/src/desul/atomics/Compare_Exchange_SYCL.hpp:12:
/kokkos/core/src/desul/atomics/SYCLConversions.hpp:33:63: error: no member named 'ONEAPI' in namespace 'sycl'
  static constexpr DESUL_SYCL_NAMESPACE::memory_order value = DESUL_SYCL_NAMESPACE::memory_order::acq_rel;
                                                              ^~~~~~~~~~~~~~~~~~~~
/kokkos/core/src/desul/atomics/Macros.hpp:38:36: note: expanded from macro 'DESUL_SYCL_NAMESPACE'
#define DESUL_SYCL_NAMESPACE sycl::ONEAPI
                             ~~~~~~^
In file included from /kokkos/core/src/impl/Kokkos_Core.cpp:45:
In file included from /kokkos/core/src/Kokkos_Core.hpp:56:
In file included from /build-test/KokkosCore_Config_DeclareBackend.hpp:47:
In file included from /kokkos/core/src/decl/Kokkos_Declare_SERIAL.hpp:49:
In file included from /kokkos/core/src/Kokkos_Serial.hpp:59:
In file included from /kokkos/core/src/Kokkos_Parallel.hpp:54:
In file included from /kokkos/core/src/Kokkos_View.hpp:500:
In file included from /kokkos/core/src/impl/Kokkos_ViewMapping.hpp:60:
In file included from /kokkos/core/src/impl/Kokkos_Atomic_View.hpp:48:
In file included from /kokkos/core/src/Kokkos_Atomic.hpp:77:
In file included from /kokkos/core/src/Kokkos_Atomics_Desul_Wrapper.hpp:6:
In file included from /kokkos/core/src/desul/atomics.hpp:14:
In file included from /kokkos/core/src/desul/atomics/Atomic_Ref.hpp:17:
In file included from /kokkos/core/src/desul/atomics/Generic.hpp:16:
In file included from /kokkos/core/src/desul/atomics/Compare_Exchange.hpp:33:
In file included from /kokkos/core/src/desul/atomics/Compare_Exchange_SYCL.hpp:12:
/kokkos/core/src/desul/atomics/SYCLConversions.hpp:37:20: error: no member named 'ONEAPI' in namespace 'sycl'
  static constexpr DESUL_SYCL_NAMESPACE::memory_order value = DESUL_SYCL_NAMESPACE::memory_order::relaxed;
                   ^~~~~~~~~~~~~~~~~~~~
/kokkos/core/src/desul/atomics/Macros.hpp:38:36: note: expanded from macro 'DESUL_SYCL_NAMESPACE'
#define DESUL_SYCL_NAMESPACE sycl::ONEAPI
                             ~~~~~~^
In file included from /kokkos/core/src/impl/Kokkos_Core.cpp:45:
In file included from /kokkos/core/src/Kokkos_Core.hpp:56:
In file included from /build-test/KokkosCore_Config_DeclareBackend.hpp:47:
In file included from /kokkos/core/src/decl/Kokkos_Declare_SERIAL.hpp:49:
In file included from /kokkos/core/src/Kokkos_Serial.hpp:59:
In file included from /kokkos/core/src/Kokkos_Parallel.hpp:54:
In file included from /kokkos/core/src/Kokkos_View.hpp:500:
In file included from /kokkos/core/src/impl/Kokkos_ViewMapping.hpp:60:
In file included from /kokkos/core/src/impl/Kokkos_Atomic_View.hpp:48:
In file included from /kokkos/core/src/Kokkos_Atomic.hpp:77:
In file included from /kokkos/core/src/Kokkos_Atomics_Desul_Wrapper.hpp:6:
In file included from /kokkos/core/src/desul/atomics.hpp:14:
In file included from /kokkos/core/src/desul/atomics/Atomic_Ref.hpp:17:
In file included from /kokkos/core/src/desul/atomics/Generic.hpp:16:
In file included from /kokkos/core/src/desul/atomics/Compare_Exchange.hpp:33:
In file included from /kokkos/core/src/desul/atomics/Compare_Exchange_SYCL.hpp:12:
/kokkos/core/src/desul/atomics/SYCLConversions.hpp:37:63: error: no member named 'ONEAPI' in namespace 'sycl'
  static constexpr DESUL_SYCL_NAMESPACE::memory_order value = DESUL_SYCL_NAMESPACE::memory_order::relaxed;
                                                              ^~~~~~~~~~~~~~~~~~~~
/kokkos/core/src/desul/atomics/Macros.hpp:38:36: note: expanded from macro 'DESUL_SYCL_NAMESPACE'
#define DESUL_SYCL_NAMESPACE sycl::ONEAPI
                             ~~~~~~^
In file included from /kokkos/core/src/impl/Kokkos_Core.cpp:45:
In file included from /kokkos/core/src/Kokkos_Core.hpp:56:
In file included from /build-test/KokkosCore_Config_DeclareBackend.hpp:47:
In file included from /kokkos/core/src/decl/Kokkos_Declare_SERIAL.hpp:49:
In file included from /kokkos/core/src/Kokkos_Serial.hpp:59:
In file included from /kokkos/core/src/Kokkos_Parallel.hpp:54:
In file included from /kokkos/core/src/Kokkos_View.hpp:500:
In file included from /kokkos/core/src/impl/Kokkos_ViewMapping.hpp:60:
In file included from /kokkos/core/src/impl/Kokkos_Atomic_View.hpp:48:
In file included from /kokkos/core/src/Kokkos_Atomic.hpp:77:
In file included from /kokkos/core/src/Kokkos_Atomics_Desul_Wrapper.hpp:6:
In file included from /kokkos/core/src/desul/atomics.hpp:14:
In file included from /kokkos/core/src/desul/atomics/Atomic_Ref.hpp:17:
In file included from /kokkos/core/src/desul/atomics/Generic.hpp:16:
In file included from /kokkos/core/src/desul/atomics/Compare_Exchange.hpp:33:
In file included from /kokkos/core/src/desul/atomics/Compare_Exchange_SYCL.hpp:12:
/kokkos/core/src/desul/atomics/SYCLConversions.hpp:44:20: error: no member named 'ONEAPI' in namespace 'sycl'
  static constexpr DESUL_SYCL_NAMESPACE::memory_scope value = DESUL_SYCL_NAMESPACE::memory_scope::work_group;
                   ^~~~~~~~~~~~~~~~~~~~
/kokkos/core/src/desul/atomics/Macros.hpp:38:36: note: expanded from macro 'DESUL_SYCL_NAMESPACE'
#define DESUL_SYCL_NAMESPACE sycl::ONEAPI
                             ~~~~~~^
In file included from /kokkos/core/src/impl/Kokkos_Core.cpp:45:
In file included from /kokkos/core/src/Kokkos_Core.hpp:56:
In file included from /build-test/KokkosCore_Config_DeclareBackend.hpp:47:
In file included from /kokkos/core/src/decl/Kokkos_Declare_SERIAL.hpp:49:
In file included from /kokkos/core/src/Kokkos_Serial.hpp:59:
In file included from /kokkos/core/src/Kokkos_Parallel.hpp:54:
In file included from /kokkos/core/src/Kokkos_View.hpp:500:
In file included from /kokkos/core/src/impl/Kokkos_ViewMapping.hpp:60:
In file included from /kokkos/core/src/impl/Kokkos_Atomic_View.hpp:48:
In file included from /kokkos/core/src/Kokkos_Atomic.hpp:77:
In file included from /kokkos/core/src/Kokkos_Atomics_Desul_Wrapper.hpp:6:
In file included from /kokkos/core/src/desul/atomics.hpp:14:
In file included from /kokkos/core/src/desul/atomics/Atomic_Ref.hpp:17:
In file included from /kokkos/core/src/desul/atomics/Generic.hpp:16:
In file included from /kokkos/core/src/desul/atomics/Compare_Exchange.hpp:33:
In file included from /kokkos/core/src/desul/atomics/Compare_Exchange_SYCL.hpp:12:
/kokkos/core/src/desul/atomics/SYCLConversions.hpp:44:63: error: no member named 'ONEAPI' in namespace 'sycl'
  static constexpr DESUL_SYCL_NAMESPACE::memory_scope value = DESUL_SYCL_NAMESPACE::memory_scope::work_group;
                                                              ^~~~~~~~~~~~~~~~~~~~
/kokkos/core/src/desul/atomics/Macros.hpp:38:36: note: expanded from macro 'DESUL_SYCL_NAMESPACE'
#define DESUL_SYCL_NAMESPACE sycl::ONEAPI
                             ~~~~~~^
In file included from /kokkos/core/src/impl/Kokkos_Core.cpp:45:
In file included from /kokkos/core/src/Kokkos_Core.hpp:56:
In file included from /build-test/KokkosCore_Config_DeclareBackend.hpp:47:
In file included from /kokkos/core/src/decl/Kokkos_Declare_SERIAL.hpp:49:
In file included from /kokkos/core/src/Kokkos_Serial.hpp:59:
In file included from /kokkos/core/src/Kokkos_Parallel.hpp:54:
In file included from /kokkos/core/src/Kokkos_View.hpp:500:
In file included from /kokkos/core/src/impl/Kokkos_ViewMapping.hpp:60:
In file included from /kokkos/core/src/impl/Kokkos_Atomic_View.hpp:48:
In file included from /kokkos/core/src/Kokkos_Atomic.hpp:77:
In file included from /kokkos/core/src/Kokkos_Atomics_Desul_Wrapper.hpp:6:
In file included from /kokkos/core/src/desul/atomics.hpp:14:
In file included from /kokkos/core/src/desul/atomics/Atomic_Ref.hpp:17:
In file included from /kokkos/core/src/desul/atomics/Generic.hpp:16:
In file included from /kokkos/core/src/desul/atomics/Compare_Exchange.hpp:33:
In file included from /kokkos/core/src/desul/atomics/Compare_Exchange_SYCL.hpp:12:
/kokkos/core/src/desul/atomics/SYCLConversions.hpp:48:20: error: no member named 'ONEAPI' in namespace 'sycl'
  static constexpr DESUL_SYCL_NAMESPACE::memory_scope value = DESUL_SYCL_NAMESPACE::memory_scope::device;
                   ^~~~~~~~~~~~~~~~~~~~
/kokkos/core/src/desul/atomics/Macros.hpp:38:36: note: expanded from macro 'DESUL_SYCL_NAMESPACE'
#define DESUL_SYCL_NAMESPACE sycl::ONEAPI
                             ~~~~~~^
In file included from /kokkos/core/src/impl/Kokkos_Core.cpp:45:
In file included from /kokkos/core/src/Kokkos_Core.hpp:56:
In file included from /build-test/KokkosCore_Config_DeclareBackend.hpp:47:
In file included from /kokkos/core/src/decl/Kokkos_Declare_SERIAL.hpp:49:
In file included from /kokkos/core/src/Kokkos_Serial.hpp:59:
In file included from /kokkos/core/src/Kokkos_Parallel.hpp:54:
In file included from /kokkos/core/src/Kokkos_View.hpp:500:
In file included from /kokkos/core/src/impl/Kokkos_ViewMapping.hpp:60:
In file included from /kokkos/core/src/impl/Kokkos_Atomic_View.hpp:48:
In file included from /kokkos/core/src/Kokkos_Atomic.hpp:77:
In file included from /kokkos/core/src/Kokkos_Atomics_Desul_Wrapper.hpp:6:
In file included from /kokkos/core/src/desul/atomics.hpp:14:
In file included from /kokkos/core/src/desul/atomics/Atomic_Ref.hpp:17:
In file included from /kokkos/core/src/desul/atomics/Generic.hpp:16:
In file included from /kokkos/core/src/desul/atomics/Compare_Exchange.hpp:33:
In file included from /kokkos/core/src/desul/atomics/Compare_Exchange_SYCL.hpp:12:
/kokkos/core/src/desul/atomics/SYCLConversions.hpp:48:63: error: no member named 'ONEAPI' in namespace 'sycl'
  static constexpr DESUL_SYCL_NAMESPACE::memory_scope value = DESUL_SYCL_NAMESPACE::memory_scope::device;
                                                              ^~~~~~~~~~~~~~~~~~~~
/kokkos/core/src/desul/atomics/Macros.hpp:38:36: note: expanded from macro 'DESUL_SYCL_NAMESPACE'
#define DESUL_SYCL_NAMESPACE sycl::ONEAPI
                             ~~~~~~^
In file included from /kokkos/core/src/impl/Kokkos_Core.cpp:45:
In file included from /kokkos/core/src/Kokkos_Core.hpp:56:
In file included from /build-test/KokkosCore_Config_DeclareBackend.hpp:47:
In file included from /kokkos/core/src/decl/Kokkos_Declare_SERIAL.hpp:49:
In file included from /kokkos/core/src/Kokkos_Serial.hpp:59:
In file included from /kokkos/core/src/Kokkos_Parallel.hpp:54:
In file included from /kokkos/core/src/Kokkos_View.hpp:500:
In file included from /kokkos/core/src/impl/Kokkos_ViewMapping.hpp:60:
In file included from /kokkos/core/src/impl/Kokkos_Atomic_View.hpp:48:
In file included from /kokkos/core/src/Kokkos_Atomic.hpp:77:
In file included from /kokkos/core/src/Kokkos_Atomics_Desul_Wrapper.hpp:6:
In file included from /kokkos/core/src/desul/atomics.hpp:14:
In file included from /kokkos/core/src/desul/atomics/Atomic_Ref.hpp:17:
In file included from /kokkos/core/src/desul/atomics/Generic.hpp:16:
In file included from /kokkos/core/src/desul/atomics/Compare_Exchange.hpp:33:
In file included from /kokkos/core/src/desul/atomics/Compare_Exchange_SYCL.hpp:12:
/kokkos/core/src/desul/atomics/SYCLConversions.hpp:52:20: error: no member named 'ONEAPI' in namespace 'sycl'
  static constexpr DESUL_SYCL_NAMESPACE::memory_scope value = DESUL_SYCL_NAMESPACE::memory_scope::system;
                   ^~~~~~~~~~~~~~~~~~~~
/kokkos/core/src/desul/atomics/Macros.hpp:38:36: note: expanded from macro 'DESUL_SYCL_NAMESPACE'
#define DESUL_SYCL_NAMESPACE sycl::ONEAPI
                             ~~~~~~^
In file included from /kokkos/core/src/impl/Kokkos_Core.cpp:45:
In file included from /kokkos/core/src/Kokkos_Core.hpp:56:
In file included from /build-test/KokkosCore_Config_DeclareBackend.hpp:47:
In file included from /kokkos/core/src/decl/Kokkos_Declare_SERIAL.hpp:49:
In file included from /kokkos/core/src/Kokkos_Serial.hpp:59:
In file included from /kokkos/core/src/Kokkos_Parallel.hpp:54:
In file included from /kokkos/core/src/Kokkos_View.hpp:500:
In file included from /kokkos/core/src/impl/Kokkos_ViewMapping.hpp:60:
In file included from /kokkos/core/src/impl/Kokkos_Atomic_View.hpp:48:
In file included from /kokkos/core/src/Kokkos_Atomic.hpp:77:
In file included from /kokkos/core/src/Kokkos_Atomics_Desul_Wrapper.hpp:6:
In file included from /kokkos/core/src/desul/atomics.hpp:14:
In file included from /kokkos/core/src/desul/atomics/Atomic_Ref.hpp:17:
In file included from /kokkos/core/src/desul/atomics/Generic.hpp:16:
In file included from /kokkos/core/src/desul/atomics/Compare_Exchange.hpp:33:
In file included from /kokkos/core/src/desul/atomics/Compare_Exchange_SYCL.hpp:12:
/kokkos/core/src/desul/atomics/SYCLConversions.hpp:52:63: error: no member named 'ONEAPI' in namespace 'sycl'
  static constexpr DESUL_SYCL_NAMESPACE::memory_scope value = DESUL_SYCL_NAMESPACE::memory_scope::system;
                                                              ^~~~~~~~~~~~~~~~~~~~
/kokkos/core/src/desul/atomics/Macros.hpp:38:36: note: expanded from macro 'DESUL_SYCL_NAMESPACE'
#define DESUL_SYCL_NAMESPACE sycl::ONEAPI
                             ~~~~~~^
In file included from /kokkos/core/src/impl/Kokkos_Core.cpp:45:
In file included from /kokkos/core/src/Kokkos_Core.hpp:56:
In file included from /build-test/KokkosCore_Config_DeclareBackend.hpp:47:
In file included from /kokkos/core/src/decl/Kokkos_Declare_SERIAL.hpp:49:
In file included from /kokkos/core/src/Kokkos_Serial.hpp:59:
In file included from /kokkos/core/src/Kokkos_Parallel.hpp:54:
In file included from /kokkos/core/src/Kokkos_View.hpp:500:
In file included from /kokkos/core/src/impl/Kokkos_ViewMapping.hpp:60:
In file included from /kokkos/core/src/impl/Kokkos_Atomic_View.hpp:48:
In file included from /kokkos/core/src/Kokkos_Atomic.hpp:77:
In file included from /kokkos/core/src/Kokkos_Atomics_Desul_Wrapper.hpp:6:
In file included from /kokkos/core/src/desul/atomics.hpp:14:
In file included from /kokkos/core/src/desul/atomics/Atomic_Ref.hpp:17:
In file included from /kokkos/core/src/desul/atomics/Generic.hpp:16:
In file included from /kokkos/core/src/desul/atomics/Compare_Exchange.hpp:33:
/kokkos/core/src/desul/atomics/Compare_Exchange_SYCL.hpp:22:3: error: no member named 'ONEAPI' in namespace 'sycl'
  DESUL_SYCL_NAMESPACE::atomic_fence(DesulToSYCLMemoryOrder<MemoryOrder>::value,
  ^~~~~~~~~~~~~~~~~~~~
/kokkos/core/src/desul/atomics/Macros.hpp:38:36: note: expanded from macro 'DESUL_SYCL_NAMESPACE'
#define DESUL_SYCL_NAMESPACE sycl::ONEAPI
                             ~~~~~~^
In file included from /kokkos/core/src/impl/Kokkos_Core.cpp:45:
In file included from /kokkos/core/src/Kokkos_Core.hpp:56:
In file included from /build-test/KokkosCore_Config_DeclareBackend.hpp:47:
In file included from /kokkos/core/src/decl/Kokkos_Declare_SERIAL.hpp:49:
In file included from /kokkos/core/src/Kokkos_Serial.hpp:59:
In file included from /kokkos/core/src/Kokkos_Parallel.hpp:54:
In file included from /kokkos/core/src/Kokkos_View.hpp:500:
In file included from /kokkos/core/src/impl/Kokkos_ViewMapping.hpp:60:
In file included from /kokkos/core/src/impl/Kokkos_Atomic_View.hpp:48:
In file included from /kokkos/core/src/Kokkos_Atomic.hpp:77:
In file included from /kokkos/core/src/Kokkos_Atomics_Desul_Wrapper.hpp:6:
In file included from /kokkos/core/src/desul/atomics.hpp:14:
In file included from /kokkos/core/src/desul/atomics/Atomic_Ref.hpp:17:
In file included from /kokkos/core/src/desul/atomics/Generic.hpp:16:
In file included from /kokkos/core/src/desul/atomics/Compare_Exchange.hpp:33:
/kokkos/core/src/desul/atomics/Compare_Exchange_SYCL.hpp:30:3: error: no member named 'ONEAPI' in namespace 'sycl'
  DESUL_SYCL_NAMESPACE::atomic_ref<
  ^~~~~~~~~~~~~~~~~~~~
/kokkos/core/src/desul/atomics/Macros.hpp:38:36: note: expanded from macro 'DESUL_SYCL_NAMESPACE'
#define DESUL_SYCL_NAMESPACE sycl::ONEAPI
                             ~~~~~~^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
20 errors generated.
core/src/CMakeFiles/kokkoscore.dir/build.make:89: recipe for target 'core/src/CMakeFiles/kokkoscore.dir/impl/Kokkos_Core.cpp.o' failed
make[2]: *** [core/src/CMakeFiles/kokkoscore.dir/impl/Kokkos_Core.cpp.o] Error 1
make[2]: Leaving directory '/build-test'
CMakeFiles/Makefile2:1318: recipe for target 'core/src/CMakeFiles/kokkoscore.dir/all' failed
make[1]: *** [core/src/CMakeFiles/kokkoscore.dir/all] Error 2
make[1]: Leaving directory '/build-test'
Makefile:145: recipe for target 'all' failed
make: *** [all] Error 2
pgrete commented 2 years ago

Thanks for your quick reply, @luisfponce

What version of intel/oneapi-basekit image are you using? I supose you are using latest (tag 2021.4-devel-ubuntu18.04) right?

I just pulled intel/oneapi-basekit rather than intel/oneapi-basekit:2021.4-devel-ubuntu18.04. They result in different containers:

CONTAINER ID   IMAGE                                           COMMAND                  CREATED         STATUS         PORTS     NAMES
b097662eca7d   intel/oneapi-basekit:2021.4-devel-ubuntu18.04   "bash -c 'source /ro…"   3 minutes ago   Up 3 minutes             nifty_mclean
e51c18972004   intel/oneapi-basekit                            "/bin/bash"              21 hours ago    Up 21 hours              zealous_lichterman

Using the container you tried, I was able to reproduce the issue you saw. I assume that the (newer) version I tried contains some fixed in the compiler that are necessary for Kokkos. In intel/oneapi-basekit:2021.4-devel-ubuntu18.04:

$ dpcpp --version
Intel(R) oneAPI DPC++/C++ Compiler 2021.4.0 (2021.4.0.20210924)

versus intel/oneapi-basekit:

$ dpcpp --version
Intel(R) oneAPI DPC++/C++ Compiler 2022.0.0 (2022.0.0.20211123)

I downloaded intel/oneapi-basekit:2021.4-devel-ubuntu18.04 and it does include llvm-foreach at /opt/intel/oneapi/compiler/2022.0.1/linux/bin-llvm/llvm-foreach, are you observing this file too?

Yes, the "newer" container also contains llvm-foreach though it's by default not in the PATH as explained in the README in /opt/intel/oneapi/compiler/2022.0.1/linux/bin-llvm. Adding that directory to the PATH also didn't fix the issue:

root@e4949db90b2b:/# which llvm-foreach
root@e4949db90b2b:/# export PATH=/opt/intel/oneapi/compiler/2022.0.1/linux/bin-llvm:$PATH
root@e4949db90b2b:/# which llvm-foreach
/opt/intel/oneapi/compiler/2022.0.1/linux/bin-llvm/llvm-foreach
root@e4949db90b2b:/# cmake -Bbuild-test -DKokkos_ARCH_INTEL_GEN12LP=ON -DKokkos_ENABLE_SYCL=ON -DCMAKE_CXX_STANDARD=17 -DCMAKE_CXX_COMPILER=dpcpp -DKokkos_ENABLE_EXAMPLES=ON kokkos
-- Setting default Kokkos CXX standard to 17
-- The CXX compiler identification is IntelLLVM 2022.0.0
[...]
root@e4949db90b2b:/# cmake --build build-test --verbose
/usr/local/bin/cmake -S/kokkos -B/build-test --check-build-system CMakeFiles/Makefile.cmake 0
[...]
[ 44%] Building CXX object example/query_device/CMakeFiles/KokkosExample_query_device.dir/query_device.cpp.o
cd /build-test/example/query_device && /opt/intel/oneapi/compiler/2022.0.1/linux/bin/dpcpp -DKOKKOS_DEPENDENCE -I/build-test/example/query_device -I/kokkos/example/query_device -I/build-test -I/build-test/core/src -I/kokkos/core/src -I/build-test/containers/src -I/kokkos/containers/src -I/build-test/algorithms/src -I/kokkos/algorithms/src -O2 -g -DNDEBUG -fsycl -fno-sycl-id-queries-fit-in-int -fsycl-dead-args-optimization -fsycl-unnamed-lambda -fsycl-targets=spir64_gen -Xsycl-target-backend "-device gen12lp" -MD -MT example/query_device/CMakeFiles/KokkosExample_query_device.dir/query_device.cpp.o -MF CMakeFiles/KokkosExample_query_device.dir/query_device.cpp.o.d -o CMakeFiles/KokkosExample_query_device.dir/query_device.cpp.o -c /kokkos/example/query_device/query_device.cpp
[ 46%] Linking CXX executable KokkosExample_query_device
cd /build-test/example/query_device && /usr/local/bin/cmake -E cmake_link_script CMakeFiles/KokkosExample_query_device.dir/link.txt --verbose=1
/opt/intel/oneapi/compiler/2022.0.1/linux/bin/dpcpp -O2 -g -DNDEBUG -DKOKKOS_DEPENDENCE -fsycl -fno-sycl-id-queries-fit-in-int -fsycl-dead-args-optimization -fsycl-targets=spir64_gen -Xsycl-target-backend "-device gen12lp" CMakeFiles/KokkosExample_query_device.dir/query_device.cpp.o -o KokkosExample_query_device  ../../containers/src/libkokkoscontainers.a ../../core/src/libkokkoscore.a /usr/lib/x86_64-linux-gnu/libdl.so 
llvm-foreach: No such file or directory
dpcpp: error: gen compiler command failed with exit code 1 (use -v to see invocation)
example/query_device/CMakeFiles/KokkosExample_query_device.dir/build.make:99: recipe for target 'example/query_device/KokkosExample_query_device' failed
make[2]: *** [example/query_device/KokkosExample_query_device] Error 1
make[2]: Leaving directory '/build-test'
CMakeFiles/Makefile2:1371: recipe for target 'example/query_device/CMakeFiles/KokkosExample_query_device.dir/all' failed
make[1]: *** [example/query_device/CMakeFiles/KokkosExample_query_device.dir/all] Error 2
make[1]: Leaving directory '/build-test'
Makefile:145: recipe for target 'all' failed
make: *** [all] Error 2

I'm seeing you are using kokkos's develop branch, have you tried the latest stable release of it (e.g. tag 3.5.00)?

The develop branch contains several fixes for the sycl backend and is ahead of the stable release. Still I tried the stable release and it doesn't compile (whereas with develop the problem is only in the linking step).

tingleby commented 7 months ago

Hi @pgrete, going to close this ticket, please comment if you'd like to re-open it.