"No cuda device detected, fallback to CPU!" despite cuda functioning elsewhere?

[Kelztergit]

"No GPU being used. Careful, inference might be very slow!"

For whatever reason, this one doesn't seem to play along with my cuda/gpu (4070ti)

Invoke AI (an image generator with webui) and Oobabooga (text generator with the same kind of webui?) have no troubles running on the gpu on this system, and they use cuda as well, so i can't believe cuda is not setup correctly, perhaps it's a version issue?

So since its just this one having some issues, I don't have much of an idea of what to share though to help debug it so i'll just share this output for nvcc version

F:\AItools\AudioGens\bark-gui> nvcc -V nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2023 NVIDIA Corporation Built on Mon_Apr__3_17:36:15_Pacific_Daylight_Time_2023 Cuda compilation tools, release 12.1, V12.1.105 Build cuda_12.1.r12.1/compiler.32688072_0

and nvidia-smi minus the processes as i doubt they matter here PS F:\AItools\AudioGens\bark-gui> nvidia-smi Mon May 1 01:00:00 2023

+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.50                 Driver Version: 535.50       CUDA Version: 12.1     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                     TCC/WDDM  | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  NVIDIA GeForce RTX 4070 Ti   WDDM  | 00000000:01:00.0  On |                  N/A |
|  0%   30C    P8               5W / 285W |    615MiB / 12282MiB |      5%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+

The main difference i can think of is the methods of installation, although similar, this one had the fewest steps and was fastest, therefore it's possible it's not as thorough potentially? if you have an answer for this behaviour, perhaps it would be nice to add a short section on setting up cuda to work with it as nobody seems to have anything written up on it on any fork of bark.

don't hesitate to ask for further information i'm willing to help debug this as it would benefit all if this kind of issue is available for all to see and learn from

[Kelztergit]

Ok i think i've confirmed it is indeed some issue with the version, I uninstalled pytorch and reinstalled it with cuda 11.7 pip uninstall torch pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu117 This is probably enough, however just to be sure i did also uninstall the 12.1 toolkit i had grabbed on developer.nvidia.com and installed 11.7, however i think this is unnecessary/overkill since pytorch sets up cuda anyway

my nvidia-smi still says 12.1 but that's hopefully not going to bite me later, might change when i restart, i thought i uninstalled it well enough... but oh well now this one works without issues for me atleast

no warnings upon opening or generating clips 💪

[meanin2]

I was having the same issue, I'll try this. Thanks!

[MatthewJadesphere]

I'm having this issue and this solution didn't seem to work for me. It's still returning "No GPU being used. Careful, inference might be very slow!" and "No CUDA detected, fallback to CPU!"

Any idea what the issue could be?

[C0untFloyd]

1. Do you have a CUDA enabled Nvidia GPU and run this without a Conda/Mamba Environment?

You could try forcing a reinstall of CUDA by typing this into the cmdline: pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118 --force-reinstall

2. Are you perhaps running this inside a conda environment (or used the 1-click windows installer for this) and forgot to install torch or answered the question for a NVIDIA GPU with no?

Either activate your conda env and try the above cmd or try re-installing everything through the 1-click installer.

[Manuel-Materazzo]

I can confirm that activating conda env and force-reinstalling cuda works. Also as additional debug step you can do:

>>> import torch
>>> torch.cuda.is_available()
True
>>> torch.version.cuda
'11.8'

if torch.version.cuda returns None conda installed the CPU version of torch.

[MatthewJadesphere]

It looks like I eventually got it to start in GPU mode after reinstalling everything several times over.