Getting error running rust using cargo: no cuda-capable device even with nvcc and nvidia-smi working

[dewrama]

Due diligence

• [X] I have done my due diligence in trying to find the answer myself.

Topic

The Rust implementation

Question

I have RTX 4060 running WSL. I have drivers installed and working for ollama and other AI applications. I also checked nvidia-smi and nvcc. But it still cannot find cuda device. Am I doing something wrong?

Bash log

CUDA_COMPUTE_CAP=86 cargo run --features cuda --bin moshi-backend -r -- --config moshi-backend/config-q8.json standalone Finished release profile [optimized + debuginfo] target(s) in 3.70s Running target/release/moshi-backend --config moshi-backend/config-q8.json standalone 2024-09-29T18:27:20.052758Z INFO moshi_backend: build_info=BuildInfo { build_timestamp: "2024-09-22T23:05:21.856959080Z", build_date: "2024-09-22", git_branch: "main", git_timestamp: "2024-09-21T17:30:23.000000000+02:00", git_date: "2024-09-21", git_hash: "3e3e573b28a1d1d6be084185e1a2e6e550c1ddcf", git_describe: "3e3e573", rustc_host_triple: "x86_64-unknown-linux-gnu", rustc_version: "1.81.0", cargo_target_triple: "x86_64-unknown-linux-gnu" } 2024-09-29T18:27:20.052859Z INFO moshi_backend: starting process with pid 54395 2024-09-29T18:27:20.055498Z INFO hf_hub: Token file not found "/home/jackson/.cache/huggingface/token"

2024-09-29T18:27:20.152210Z INFO hf_hub: Token file not found "/home/jackson/.cache/huggingface/token"

Error: DriverError(CUDA_ERROR_NO_DEVICE, "no CUDA-capable device is detected")

[dewrama]

+-----------------------------------------------------------------------------------------+ | NVIDIA-SMI 560.35.03 Driver Version: 561.09 CUDA Version: 12.6 | |-----------------------------------------+------------------------+----------------------+ | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |=========================================+========================+======================| | 0 NVIDIA GeForce RTX 4060 ... On | 00000000:01:00.0 Off | N/A | | N/A 51C P3 15W / 57W | 0MiB / 8188MiB | 0% Default | | | | N/A | +-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=========================================================================================| | No running processes found | +-----------------------------------------------------------------------------------------+