Closed KrisThielemans closed 1 month ago
@casperdcl set up a self hosted runner for CIL's CI where GPU is required. Similar thing could be done here, I believe.
Short of that, we run nvidia-smi
during pytest
to see if the machine where the tests run has a GPU that works:
try:
subprocess.check_output('nvidia-smi')
has_nvidia = True
except:
if os.environ.get("TESTS_FORCE_GPU", ""):
raise ImportError
has_nvidia = False
GitHub Actions doesn't have the CUDA run-time, so testing the CUDA RDP fails with
sirf.Utilities.error: ??? "'\\nERROR: CUDA failed to allocate memory in compute_gradient kernel execution: CUDA driver version is insufficient for CUDA runtime version\\n'
This occurs in thebuild (gpu)
docker job of the SB, see e.g. https://github.com/SyneRBI/SIRF-SuperBuild/actions/runs/9865000620/job/27241074266#step:9:29413Actual relevant lines are https://github.com/SyneRBI/SIRF/blob/cb8a9981b142006c3b01cf9264f51882afdad12a/src/xSTIR/pSTIR/tests/tests_qp_lc_rdp.py#L64-L65
In STIR, I've disabled the tests by checking in CMake if
nvida-smi
is found https://github.com/UCL/STIR/blob/09638a9f01ff674118f1d584dbeba604e2435bf9/CMakeLists.txt#L209-L213 and then passing an option to the relevant test https://github.com/UCL/STIR/blob/09638a9f01ff674118f1d584dbeba604e2435bf9/src/recon_test/CMakeLists.txt#L152-L154For SIRF, we're using pytest. @casperdcl any suggestions