Configuration step in `build_cudaq.sh` intermittently hangs

boschmitt commented 6 days ago

Required prerequisites

[x] Consult the security policy. If reporting a security vulnerability, do not report the bug using this form. Use the process described in the policy to report the issue.
[x] Make sure you've read the documentation. Your issue may be addressed there.
[x] Search the issue tracker to verify that this hasn't already been reported. +1 or comment there if it has.
[ ] If possible, make a PR with a failing test to give us a starting point to work on!

Describe the bug

When investigating #2409, I noticed that sometimes the configuration step will hang when trying to configure cpr and check whether address sanitizers are available.

root@d115d51496d2:/workspace# bash scripts/build_cudaq.sh -v
CUDA version 12.0 detected.
Using cuQuantum installation in /usr/local/lib/python3.10/dist-packages/cuquantum.
Using cuTensor installation in /usr/local/lib/python3.10/dist-packages/cutensor/.
Configuring nvq++ to use the lld linker by default.
Preparing CUDA-Q build with LLVM installation in /usr/local/llvm...
-- The CXX compiler identification is GNU 11.4.0
-- The C compiler identification is GNU 11.4.0
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/local/llvm/bootstrap/cxx - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/local/llvm/bootstrap/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Submodule update
Skipping submodule 'tpls/llvm'
-- CUDA Quantum static linking disabled.
-- Found ZLIB: /usr/local/zlib/lib/libz.a (found version "1.3.#define ZLIB_VERSION "1.3"")
-- Using LLVMConfig.cmake in: /usr/local/llvm/lib/cmake/llvm
-- Using ClangConfig.cmake in: /usr/local/llvm/lib/cmake/clang
-- Using MLIRConfig.cmake in: /usr/local/llvm/lib/cmake/mlir
-- Linker detection: GNU ld
-- Build type is Release
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- Found OpenSSL: /usr/local/openssl/lib64/libcrypto.a (found version "3.3.1")
-- Found MPI_CXX: /usr/local/openmpi/lib/libmpi.so (found version "3.1")
-- Found MPI: TRUE (found version "3.1") found components: CXX
-- MPI CXX Found: /usr/local/openmpi/bin/mpiexec
-- Building default MPI Comm plugin
-- Module support is disabled.
-- Version: 8.1.2
-- Build type: Release
-- CXX_STANDARD: 20
-- Performing Test has_std_20_flag
-- Performing Test has_std_20_flag - Success
-- Performing Test has_std_2a_flag
-- Performing Test has_std_2a_flag - Success
-- Required features: cxx_variadic_templates
-- Build spdlog: 1.11.0
-- Build type: Release
-- Found CURL: /usr/local/curl/lib/libcurl.a (found version "8.5.0") found components: HTTP HTTPS
-- Using system curl for CPR.
-- C++ Requests CMake Options
-- =======================================================
--   CPR_GENERATE_COVERAGE: OFF
--   CPR_CURL_NOSIGNAL: OFF
--   CPR_USE_SYSTEM_GTEST: OFF
--   CPR_FORCE_USE_SYSTEM_CURL: TRUE
--   CPR_ENABLE_SSL: ON
--   CPR_FORCE_OPENSSL_BACKEND: ON
--   CPR_FORCE_WINSSL_BACKEND: OFF
--   CPR_FORCE_DARWINSSL_BACKEND: OFF
--   CPR_FORCE_MBEDTLS_BACKEND: OFF
--   CPR_ENABLE_LINTING: OFF
--   CPR_ENABLE_CPPCHECK: OFF
--   CPR_BUILD_TESTS: OFF
--   CPR_BUILD_TESTS_SSL: OFF
-- =======================================================
-- Performing Test THREAD_SANITIZER_AVAILABLE
-- Performing Test THREAD_SANITIZER_AVAILABLE - Failed
-- Performing Test ADDRESS_SANITIZER_AVAILABLE

Steps to reproduce the bug

Same as #2409 and running build_cudaq.sh a few times.

Expected behavior

It shouldn't hang.

Is this a regression? If it is, put the last known working version (or commit) here.

Not a regression

Environment

CUDA-Q version: main (39a46e62ccc85922958925441c0e2e489eb27a83)
Image: cuda-quantum-devdeps:ext-cu12.0-gcc11-main

Suggestions

No response

amccaskey commented 6 days ago

I've seen this happen just from cmake, apart from the build script.

vtomole commented 6 days ago

I'm running into this issue as well.

sacpis commented 6 days ago

I have been having this issue on Ubuntu 22.04 since last March. On running it a few times (usually three) it works for me.

NVIDIA / cuda-quantum