intel / llvm

Intel staging area for llvm.org contribution. Home for Intel LLVM-based projects.
Other
1.23k stars 736 forks source link

Graph/RecordReplay/usm_fill.cpp timeout on CUDA CI for unrelated changes #15000

Open steffenlarsen opened 2 months ago

steffenlarsen commented 2 months ago

Describe the bug

The Graph/RecordReplay/usm_fill.cpp test has been observed to timeout in CUDA CI for unrelated changes. For example, see https://github.com/intel/llvm/pull/14985.

TIMEOUT: SYCL :: Graph/RecordReplay/usm_fill.cpp (1953 of 2130)
******************** TEST 'SYCL :: Graph/RecordReplay/usm_fill.cpp' FAILED ********************
Exit Code: -9
Timeout: Reached timeout of 600 seconds

Command Output (stdout):
--
# RUN: at line 1
.../llvm/toolchain/bin//clang++  -Werror  -fsycl -fsycl-targets=nvptx64-nvidia-cuda  .../llvm/llvm/sycl/test-e2e/Graph/RecordReplay/usm_fill.cpp -o .../llvm/build-e2e/Graph/RecordReplay/Output/usm_fill.cpp.tmp.out
# executed command: .../llvm/toolchain/bin//clang++ -Werror -fsycl -fsycl-targets=nvptx64-nvidia-cuda .../llvm/llvm/sycl/test-e2e/Graph/RecordReplay/usm_fill.cpp -o .../llvm/build-e2e/Graph/RecordReplay/Output/usm_fill.cpp.tmp.out
# note: command had no output on stdout or stderr
# RUN: at line 2
env SYCL_PI_CUDA_ENABLE_IMAGE_SUPPORT=1 ONEAPI_DEVICE_SELECTOR=cuda:gpu  .../llvm/build-e2e/Graph/RecordReplay/Output/usm_fill.cpp.tmp.out
# executed command: env SYCL_PI_CUDA_ENABLE_IMAGE_SUPPORT=1 ONEAPI_DEVICE_SELECTOR=cuda:gpu .../llvm/build-e2e/Graph/RecordReplay/Output/usm_fill.cpp.tmp.out
# note: command had no output on stdout or stderr
# error: command failed with exit status: -9
# error: command reached timeout: True

--

To reproduce

  1. Include a code snippet that is as short as possible
  2. Specify the command which should be used to compile the program
  3. Specify the command which should be used to launch the program
  4. Indicate what is wrong and what was expected

Environment

Additional context

No response

EwanC commented 2 months ago

Related issue for Graph E2E tests timing out in CI https://github.com/intel/llvm/issues/14852