The recent switch to lazy loading by default in CTK 12.2 seems to have broken the async benchmarks. This can be reproduced by nvbench.example.axes. The deadlock can be fixed by CUDA_MODULE_LOADING=EAGER. We should incorporate this information into the error message or set the variable ourselves.
We likely want eager loads by default anyway to make sure that lazy loads aren't affecting measurements. Let's look into defining that var from the NVBench main implementation.
The recent switch to lazy loading by default in CTK 12.2 seems to have broken the async benchmarks. This can be reproduced by
nvbench.example.axes
. The deadlock can be fixed byCUDA_MODULE_LOADING=EAGER
. We should incorporate this information into the error message or set the variable ourselves.