search

CHIP-SPV / chipStar

chipStar is a tool for compiling and running HIP/CUDA on SPIR-V via OpenCL or Level Zero APIs.
Other
182 stars 29 forks source link

HIP-SYCL interop issues with igpu #753

Closed pvelesko closed 6 months ago

pvelesko commented 7 months ago

igpu + MKL 2023.2.3 

list(APPEND IGPU_LEVEL0_BASE_FAILED_TESTS "hip_sycl_interop") # Timeout Using MKL 2023.2.3 
list(APPEND IGPU_LEVEL0_BASE_FAILED_TESTS "hip_sycl_interop_no_buffers") # Timeout Using MKL 2023.2.3

igpu + MKL 2024.0.0 

list(APPEND IGPU_LEVEL0_BASE_FAILED_TESTS "hip_sycl_interop") # pure virtual method called
list(APPEND IGPU_LEVEL0_BASE_FAILED_TESTS "hip_sycl_interop_no_buffers") # pure virtual method called

pure virtual method called callstack:

level-zero/igpu + MKL 2024 throws pure virtual method called @Sarbojit2019

dgpu_opencl_make_check_result.txt: PASS
igpu_opencl_make_check_result.txt: PASS
igpu_level0_reg_make_check_result.txt: FAIL
    499 - hip_sycl_interop (Subprocess aborted)
    500 - hip_sycl_interop_no_buffers (Subprocess aborted)
dgpu_level0_reg_make_check_result.txt: FAIL
    499 - hip_sycl_interop (SEGFAULT)
    500 - hip_sycl_interop_no_buffers (SEGFAULT)
dgpu_level0_imm_make_check_result.txt: FAIL
    500 - hip_sycl_interop (SEGFAULT)
    501 - hip_sycl_interop_no_buffers (SEGFAULT)
 Starting program: /space/pvelesko/chipStar/test-mkl/build/samples/hip_sycl_interop_no_buffers/hip_sycl_interop_no_buffers
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7fffc8c73640 (LWP 3797765)]
[New Thread 0x7fffb88c0640 (LWP 3797766)]
[New Thread 0x7fffb3fff640 (LWP 3797767)]
[Thread 0x7fffb88c0640 (LWP 3797766) exited]
[Thread 0x7fffb3fff640 (LWP 3797767) exited]
[New Thread 0x7fffb37fe640 (LWP 3797768)]
[New Thread 0x7fffb2ffd640 (LWP 3797769)]
[New Thread 0x7fffb3fff640 (LWP 3797770)]
[Detaching after fork from child process 3797771]
[Detaching after fork from child process 3797772]
[Detaching after fork from child process 3797773]
[Detaching after fork from child process 3797774]
[Detaching after fork from child process 3797775]
[Detaching after fork from child process 3797776]
[Detaching after fork from child process 3797777]
[Detaching after fork from child process 3797778]
[Detaching after fork from child process 3797779]
pure virtual method called
terminate called without an active exception

Thread 1 "hip_sycl_intero" received signal SIGABRT, Aborted.
0x00007fffcacb49fc in pthread_kill () from /usr/lib/x86_64-linux-gnu/libc.so.6
(gdb) bt
#0  0x00007fffcacb49fc in pthread_kill () from /usr/lib/x86_64-linux-gnu/libc.so.6
#1  0x00007fffcac60476 in raise () from /usr/lib/x86_64-linux-gnu/libc.so.6
#2  0x00007fffcac467f3 in abort () from /usr/lib/x86_64-linux-gnu/libc.so.6
#3  0x00007fffcaff0b9e in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#4  0x00007fffcaffc20c in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#5  0x00007fffcaffc277 in std::terminate() () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#6  0x00007fffcaffcfa5 in __cxa_pure_virtual () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#7  0x00007fffc8ff64a7 in L0::zeCommandListAppendLaunchKernel(_ze_command_list_handle_t*, _ze_kernel_handle_t*, _ze_group_count_t const*, _ze_event_handle_t*, unsigned int, _ze_event_handle_t**) ()
   from /space/pvelesko/install/intel-compute-runtime//neo/23.43.27642.21/lib/libze_intel_gpu.so.1
#8  0x00007ffff5e01c22 in zeCommandListAppendLaunchKernel () from /space/pvelesko/install/intel-compute-runtime//level-zero/23.43.27642.21/lib/libze_loader.so.1
#9  0x00007fffb2355c44 in urEnqueueKernelLaunch () from /space/pvelesko/install/oneapi/compiler/2024.0/lib/libpi_level_zero.so
#10 0x00007fffb23816f0 in piEnqueueKernelLaunch () from /space/pvelesko/install/oneapi/compiler/2024.0/lib/libpi_level_zero.so
#11 0x00007ffff638d891 in _pi_result sycl::_V1::detail::plugin::call_nocheck<(sycl::_V1::detail::PiApiKind)76, _pi_queue*, _pi_kernel*, unsigned long, unsigned long*, unsigned long*, unsigned long*, unsigned long, _pi_event**, _pi_event**>(_pi_queue*, _pi_kernel*, unsigned long, unsigned long*, unsigned long*, unsigned long*, unsigned long, _pi_event**, _pi_event**) const () from /space/pvelesko/install/oneapi/compiler/2024.0/lib/libsycl.so.7
#12 0x00007ffff6384184 in sycl::_V1::detail::enqueueImpKernel(std::shared_ptr<sycl::_V1::detail::queue_impl> const&, sycl::_V1::detail::NDRDescT&, std::vector<sycl::_V1::detail::ArgDesc, std::allocator<sycl::_V1::detail::ArgDesc> >&, std::shared_ptr<sycl::_V1::detail::kernel_bundle_impl> const&, std::shared_ptr<sycl::_V1::detail::kernel_impl> const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<_pi_event*, std::allocator<_pi_event*> >&, std::shared_ptr<sycl::_V1::detail::event_impl> const&, std::function<void* (sycl::_V1::detail::AccessorImplHost*)> const&, _pi_kernel_cache_config) () from /space/pvelesko/install/oneapi/compiler/2024.0/lib/libsycl.so.7
#13 0x00007ffff6389a02 in sycl::_V1::detail::ExecCGCommand::enqueueImpQueue() () from /space/pvelesko/install/oneapi/compiler/2024.0/lib/libsycl.so.7
#14 0x00007ffff6370f00 in sycl::_V1::detail::Command::enqueue(sycl::_V1::detail::EnqueueResultT&, sycl::_V1::detail::BlockingT, std::vector<sycl::_V1::detail::Command*, std::allocator<sycl::_V1::detail::Command*> >&) ()
   from /space/pvelesko/install/oneapi/compiler/2024.0/lib/libsycl.so.7
#15 0x00007ffff639aa14 in sycl::_V1::detail::Scheduler::GraphProcessor::enqueueCommand(sycl::_V1::detail::Command*, std::shared_lock<std::shared_timed_mutex>&, sycl::_V1::detail::EnqueueResultT&, std::vector<sycl::_V1::detail::Command*, std::allocator<sycl::_V1::detail::Command*> >&, sycl::_V1::detail::Command*, sycl::_V1::detail::BlockingT) () from /space/pvelesko/install/oneapi/compiler/2024.0/lib/libsycl.so.7
#16 0x00007ffff6395c7c in sycl::_V1::detail::Scheduler::enqueueCommandForCG(std::shared_ptr<sycl::_V1::detail::event_impl>, std::vector<sycl::_V1::detail::Command*, std::allocator<sycl::_V1::detail::Command*> >&, sycl::_V1::detail::BlockingT)
    () from /space/pvelesko/install/oneapi/compiler/2024.0/lib/libsycl.so.7
#17 0x00007ffff63953f8 in sycl::_V1::detail::Scheduler::addCG(std::unique_ptr<sycl::_V1::detail::CG, std::default_delete<sycl::_V1::detail::CG> >, std::shared_ptr<sycl::_V1::detail::queue_impl> const&, _pi_ext_command_buffer*, std::vector<unsigned int, std::allocator<unsigned int> > const&) () from /space/pvelesko/install/oneapi/compiler/2024.0/lib/libsycl.so.7
#18 0x00007ffff63cddb2 in sycl::_V1::handler::finalize() () from /space/pvelesko/install/oneapi/compiler/2024.0/lib/libsycl.so.7
#19 0x00007ffff6357b51 in void sycl::_V1::detail::queue_impl::finalizeHandler<sycl::_V1::handler>(sycl::_V1::handler&, sycl::_V1::detail::CG::CGTYPE const&, sycl::_V1::event&) ()
   from /space/pvelesko/install/oneapi/compiler/2024.0/lib/libsycl.so.7
#20 0x00007ffff6357551 in sycl::_V1::detail::queue_impl::submit_impl(std::function<void (sycl::_V1::handler&)> const&, std::shared_ptr<sycl::_V1::detail::queue_impl> const&, std::shared_ptr<sycl::_V1::detail::queue_impl> const&, std::shared_ptr<sycl::_V1::detail::queue_impl> const&, sycl::_V1::detail::code_location const&, std::function<void (bool, bool, sycl::_V1::event&)> const*) () from /space/pvelesko/install/oneapi/compiler/2024.0/lib/libsycl.so.7
#21 0x00007ffff63fb106 in sycl::_V1::detail::queue_impl::submit(std::function<void (sycl::_V1::handler&)> const&, std::shared_ptr<sycl::_V1::detail::queue_impl> const&, sycl::_V1::detail::code_location const&, std::function<void (bool, bool, sycl::_V1::event&)> const*) () from /space/pvelesko/install/oneapi/compiler/2024.0/lib/libsycl.so.7
#22 0x00007ffff63fb0c5 in sycl::_V1::queue::submit_impl(std::function<void (sycl::_V1::handler&)>, sycl::_V1::detail::code_location const&) () from /space/pvelesko/install/oneapi/compiler/2024.0/lib/libsycl.so.7
#23 0x00007ffff10b9ee0 in oneapi::mkl::gpu::launch_kernel_3D(int*, sycl::_V1::queue*, mkl_gpu_kernel_struct_t*, mkl_gpu_argument_t*, unsigned long*, unsigned long*, mkl_gpu_event_list_t*) ()
   from /space/pvelesko/install/oneapi/mkl/2024.0/lib/libmkl_sycl_blas.so.4
#24 0x00007ffff10afa11 in oneapi::mkl::gpu::have_binary_kernels(int*, sycl::_V1::queue*) () from /space/pvelesko/install/oneapi/mkl/2024.0/lib/libmkl_sycl_blas.so.4
#25 0x00007ffff296aaac in oneapi::mkl::gpu::mkl_blas_gpu_sgemm_driver_sycl(int*, sycl::_V1::queue*, oneapi::mkl::gpu::blas_arg_usm_t*, mkl_gpu_event_list_t*) () from /space/pvelesko/install/oneapi/mkl/2024.0/lib/libmkl_sycl_blas.so.4
#26 0x00007ffff295593b in oneapi::mkl::gpu::sgemm_sycl_internal(sycl::_V1::queue*, MKL_LAYOUT, MKL_TRANSPOSE, MKL_TRANSPOSE, long, long, long, oneapi::mkl::value_or_pointer<float>, float const*, long, float const*, long, oneapi::mkl::value_or_pointer<float>, float*, long, oneapi::mkl::blas::compute_mode, std::vector<sycl::_V1::event, std::allocator<sycl::_V1::event> > const&, long, long, long) () from /space/pvelesko/install/oneapi/mkl/2024.0/lib/libmkl_sycl_blas.so.4
#27 0x00007ffff2953f5d in oneapi::mkl::gpu::sgemm_sycl(sycl::_V1::queue*, MKL_LAYOUT, MKL_TRANSPOSE, MKL_TRANSPOSE, long, long, long, oneapi::mkl::value_or_pointer<float>, float const*, long, float const*, long, oneapi::mkl::value_or_pointer<float>, float*, long, oneapi::mkl::blas::compute_mode, std::vector<sycl::_V1::event, std::allocator<sycl::_V1::event> > const&, long, long, long) () from /space/pvelesko/install/oneapi/mkl/2024.0/lib/libmkl_sycl_blas.so.4
#28 0x00007ffff3393d22 in oneapi::mkl::blas::sgemm(sycl::_V1::queue&, MKL_LAYOUT, oneapi::mkl::transpose, oneapi::mkl::transpose, long, long, long, oneapi::mkl::value_or_pointer<float>, float const*, long, float const*, long, oneapi::mkl::value_or_pointer<float>, float*, long, oneapi::mkl::blas::compute_mode, std::vector<sycl::_V1::event, std::allocator<sycl::_V1::event> > const&) () from /space/pvelesko/install/oneapi/mkl/2024.0/lib/libmkl_sycl_blas.so.4
#29 0x00007ffff331bf2f in oneapi::mkl::blas::column_major::gemm(sycl::_V1::queue&, oneapi::mkl::transpose, oneapi::mkl::transpose, long, long, long, oneapi::mkl::value_or_pointer<float>, float const*, long, float const*, long, oneapi::mkl::value_or_pointer<float>, float*, long, oneapi::mkl::blas::compute_mode, std::vector<sycl::_V1::event, std::allocator<sycl::_V1::event> > const&) () from /space/pvelesko/install/oneapi/mkl/2024.0/lib/libmkl_sycl_blas.so.4
#30 0x00007ffff60b94e1 in oneapi::mkl::blas::column_major::gemm (queue=..., transa=oneapi::mkl::transpose::nontrans, transb=oneapi::mkl::transpose::nontrans, m=140737488299904, m@entry=10, n=0, k=0, alpha=..., a=<optimized out>,
    lda=<optimized out>, b=<optimized out>, ldb=8, beta=..., c=<optimized out>, ldc=582, dependencies=...) at /home/pvelesko/space/install/oneapi/mkl/2024.0/include/oneapi/mkl/blas/usm_decls.hpp:38
#31 onemkl_gemm (my_queue=..., A=<optimized out>, B=<optimized out>, C=C@entry=0xffffd556aa7c0000, m=m@entry=10, n=<optimized out>, k=10, ldA=10, ldB=10, ldC=10, alpha=<optimized out>, beta=-7457682)
    at /home/pvelesko/space/chipStar/test-mkl/samples/hip_sycl_interop_no_buffers/onemkl_gemm_wrapper_no_buffers/onemkl_gemm_wrapper.cpp:63
#32 0x00007ffff60ba266 in oneMKLGemmTest (nativeHandlers=<optimized out>, hip_backend=<optimized out>, A=<optimized out>, B=<optimized out>, C=<optimized out>, M=<optimized out>, N=<optimized out>, K=<optimized out>, ldA=<optimized out>,
    ldB=<optimized out>, ldC=<optimized out>, alpha=<optimized out>, beta=<optimized out>) at /home/pvelesko/space/chipStar/test-mkl/samples/hip_sycl_interop_no_buffers/onemkl_gemm_wrapper_no_buffers/onemkl_gemm_wrapper.cpp:132
#33 0x0000555555556716 in main () at /home/pvelesko/space/chipStar/test-mkl/samples/hip_sycl_interop_no_buffers/hip_sycl_interop.cpp:121