CUDA device requested but no devices found

[mpizzagalli777]

Issue Report

Please describe the issue:

Hi, I just installed Dorado on my workstation. I am running into trouble having it run successfully due to an unknown CUDA error. I ahve been unable to find a solution to this issue in other posts. Running nvidia-smi gives the following results. I am aware that we do not have the A100, but I read that the A6000 should work as well (although it might be a little slower).

Here is the output of ldd dorado

Steps to reproduce the issue:

Please list any steps to reproduce the issue.

Run environment:

• Dorado version: 0.6.1
• Dorado command: dorado basecaller hac pod5/ > ./basecalling/GB2_p22_dorado_hac.bam
• Operating system: Linux
• Hardware (CPUs, Memory, GPUs):
• Source data type (e.g., pod5 or fast5 - please note we always recommend converting to pod5 for optimal basecalling performance): pod5
• Source data location (on device or networked drive - NFS, etc.): on device

Please let me know if there is any other information that might be useful.

[tijyojwad]

Hi @mpizzagalli777 - are you able to run any other CUDA application? Can you try to run the CUDA samples from https://github.com/NVIDIA/cuda-samples ? This will help rule out if it's a setup issue or not

[mpizzagalli777]

Sorry about the messaging mess. I ran the CUDA samples and got the following error:

CUDA error at ../../../Common/helper_cuda.h:801 code=999(cudaErrorUnknown) "cudaGetDeviceCount(&device_count)"

Do you happen to know what might be causing this? I tried reinstalling CUDA toolkit (12.4). Not sure if you might know how to troubleshoot this further

[tijyojwad]

Hi, posting on the cuda samples GitHub might be helpful! I've also found from personal experience that sometimes restarting after a CUDA driver install/upgrade helps too.

I'm closing this issue for now, since it's not particular to dorado.

[mpizzagalli777]

Thanks so much for your help. In case someone else has the same issue, I followed the download instructions here: https://developer.nvidia.com/cuda-downloads And after restarting my computer, was able to successfully run the cuda-samples test.