Unable to use GPU - Githubissues

stmolivier commented 1 year ago

I am on windows 10 and Python 10 is installed. I have a NVIDIA Geforce GTX 1060 with 6GB and a I7 CPU with 32Go Ram

I have installed bark in c:\bark
I have downloaded and installed in a model folder the 6 models (pt files)
I have added XDG_CACHE_HOME to the environnment variable for the model folder and
In generation.py I modified CACHE_DIR = os.getenv("XDG_CACHE_HOME")

I tried the demo test from the readme. I launch this script python.exe -m demo.py as Administrator The audio is generated, very slowly, but I have the message : No GPU being used. Careful, inference might be very slow! I also have this message

C:\Users\meme\AppData\Roaming\Python\Python310\site-packages\torchaudio\backend\utils.py:74: UserWarning: No audio backend is available.
  warnings.warn("No audio backend is available.")

How can I make (force) the scripts to use my Nvidia GPU ? I would like to at least test if there's a big difference for the generation time. I'm not very fluent in Python :)

Thank you

5118Python commented 1 year ago

https://pytorch.org/get-started/locally/ pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu117

stmolivier commented 1 year ago

Thank you for your answer. I've already installed this and it doesn't change the outcome... Is there a parameter to pass to the script launch ? a line to add into the demo script ? A Path variable to add ?

D80-Sheffield commented 1 year ago

I have the same problem like you,it seems like you got a wrong torch version, check if torch and cuda are available, ` import torch

print(f'PyTorch version: {torch.version}') if torch.cuda.is_available(): print(f'CUDA is available，version is:{torch.version.cuda}') if torch.backends.cudnn.is_available(): print(f'cuDNN is available，version is:{torch.backends.cudnn.version()}') else: print('cuDNN Not Available') else: print('CUDA Not Available')

if torch.cuda.is_available(): a = torch.tensor(1, device='cuda') b = torch.tensor(2, device='cuda') print(a + b) else: a = torch.tensor(1) b = torch.tensor(2) print(a + b) ` if your PyTorch version is 'xxxx+cpu', use 'pip uninstall torch' first, then go to https://pytorch.org/get-started/locally/ download correct version with gpu. it may solve your problem :-)

equilibrium-liberalism commented 1 year ago

here has a snippet how to let it work on GPU with 8G VRAM. https://rewa-insights.com/t/text-to-voice/217/2

venturaEffect commented 1 year ago

I'm encountering the same problem. From what I see of the link @D80-Sheffield has provided is that the CUDA's available is 11.8, when I have CUDA 12.2 and an NVIDIA RTX 4090.

So I changed the command like this: pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu122

And when running it I get "Reuirement alreaqdy satisfied... " so this isn't the problem.

Any solution would be really appreciated.

SimBuddy commented 1 year ago

I stumbled upon something that worked for me...I installed Visual Studio, Git, Anaconda, Cuda 10.1.105 (others may work too, but 12.x didn't) and eSpeak (not sure if that is an actual requirement)

Then I ran: pip install git+https://github.com/huggingface/transformers.git pip install git+https://github.com/suno-ai/bark.git pip uninstall torch torchaudio pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu117

In the python code I had:

import scipy import os import torch

os.environ["SUNO_OFFLOAD_CPU"] = "True" os.environ["SUNO_USE_SMALL_MODELS"] = "True"

from transformers import AutoProcessor, BarkModel

processor = AutoProcessor.from_pretrained("suno/bark") model = BarkModel.from_pretrained("suno/bark") model.to('cuda')

voice_preset = "v2/en_speaker_6"

inputs = processor("Text that you want to convert to speech.", voice_preset=voice_preset)

inputs.to('cuda') audio_array = model.generate(**inputs,pad_token_id=10500) audio_array = audio_array.cpu().numpy().squeeze() sample_rate = model.generation_config.sample_rate scipy.io.wavfile.write("bark_out.wav", rate=sample_rate, data=audio_array)

Nothing worked until I added the line above, i.e. inputs.to('cuda')

After that, I could see the GPU kicking in on Task Manager. I know diddly-squat about all this, but spotted the critical code in another post somewhere else. I may be taking extra steps that are not required, but hey ho - it's working now.

mtnmecca commented 1 year ago

after hours of searching for a solution, SimBuddy has the solution - thank you!!

SimBuddy commented 1 year ago

Only hours? I was at it for days! I had all but given up when I spotted the critical line of code somewhere. Glad to be of service. P.Sent from Samsung Mobile on O2 -------- Original message --------From: mtnmecca @.> Date: 06/09/2023 21:00 (GMT+00:00) To: suno-ai/bark @.> Cc: SimBuddy @.>, Comment @.> Subject: Re: [suno-ai/bark] Unable to use GPU (Issue #342) after hours of searching for a solution, SimBuddy has the solution - thank you!!

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: @.***>

mtnmecca commented 1 year ago

Huge help! I can process so fast now on gpu. Now on to fixing TTS and Bark compatibility. Numpy 1.25.2 (bark) vs 1.22.0 (TTS). Always something.. Thanks again.

venturaEffect commented 1 year ago

Still not working...

I followed the steps. Created a file inside the parent bark folder and added the piece of code @D80-Sheffield provided. After trying to run it I got an error:

Traceback (most recent call last): File "C:\Users\zaesa\bark\for_installing_done_by_me.py", line 12, in <module> model.to('cuda') File "C:\Users\zaesa\anaconda3\Lib\site-packages\transformers\modeling_utils.py", line 2014, in to return super().to(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\zaesa\anaconda3\Lib\site-packages\torch\nn\modules\module.py", line 1145, in to return self._apply(convert) ^^^^^^^^^^^^^^^^^^^^ File "C:\Users\zaesa\anaconda3\Lib\site-packages\torch\nn\modules\module.py", line 797, in _apply module._apply(fn) File "C:\Users\zaesa\anaconda3\Lib\site-packages\torch\nn\modules\module.py", line 797, in _apply module._apply(fn) File "C:\Users\zaesa\anaconda3\Lib\site-packages\torch\nn\modules\module.py", line 820, in _apply param_applied = fn(param) ^^^^^^^^^ File "C:\Users\zaesa\anaconda3\Lib\site-packages\torch\nn\modules\module.py", line 1143, in convert return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\zaesa\anaconda3\Lib\site-packages\torch\cuda\__init__.py", line 239, in _lazy_init raise AssertionError("Torch not compiled with CUDA enabled") AssertionError: Torch not compiled with CUDA enabled

I searched everywhere and tried uninstall torch: pip uninstall torch

Installed again but with my cuda version: pip install torch torchvision torchaudio -f https://download.pytorch.org/whl/cu122/torch_stable.html Nothing, running again the script @D80-Sheffield still get the error.

No idea what is going on. Any help will be really appreciated.

SimBuddy commented 1 year ago

Unfortunately, I know next to nothing about this subject. I got lucky, so thought it might be useful. Sorry.Sent from Samsung Mobile on O2 -------- Original message --------From: Cesar Romero @.> Date: 07/09/2023 09:34 (GMT+00:00) To: suno-ai/bark @.> Cc: SimBuddy @.>, Comment @.> Subject: Re: [suno-ai/bark] Unable to use GPU (Issue #342) Still not working... I followed the steps. Created a file inside the parent bark folder and added the piece of code @D80-Sheffield provided. After trying to run it I got an error: Traceback (most recent call last): File "C:\Users\zaesa\bark\for_installing_done_by_me.py", line 12, in model.to('cuda') File "C:\Users\zaesa\anaconda3\Lib\site-packages\transformers\modeling_utils.py", line 2014, in to return super().to(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\zaesa\anaconda3\Lib\site-packages\torch\nn\modules\module.py", line 1145, in to return self._apply(convert) ^^^^^^^^^^^^^^^^^^^^ File "C:\Users\zaesa\anaconda3\Lib\site-packages\torch\nn\modules\module.py", line 797, in _apply module._apply(fn) File "C:\Users\zaesa\anaconda3\Lib\site-packages\torch\nn\modules\module.py", line 797, in _apply module._apply(fn) File "C:\Users\zaesa\anaconda3\Lib\site-packages\torch\nn\modules\module.py", line 820, in _apply param_applied = fn(param) ^^^^^^^^^ File "C:\Users\zaesa\anaconda3\Lib\site-packages\torch\nn\modules\module.py", line 1143, in convert return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\zaesa\anaconda3\Lib\site-packages\torch\cuda__init__.py", line 239, in _lazy_init raise AssertionError("Torch not compiled with CUDA enabled") AssertionError: Torch not compiled with CUDA enabled I searched everywhere and tried uninstall torch: pip uninstall torch Installed again but with my cuda version: pip install torch torchvision torchaudio -f https://download.pytorch.org/whl/cu122/torch_stable.html Nothing, running again the script @D80-Sheffield still get the error. No idea what is going on. Any help will be really appreciated.

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: @.***>

BrawnyAi commented 9 months ago

I stumbled upon something that worked for me...I installed Visual Studio, Git, Anaconda, Cuda 10.1.105 (others may work too, but 12.x didn't) and eSpeak (not sure if that is an actual requirement)

Then I ran: pip install git+https://github.com/huggingface/transformers.git pip install git+https://github.com/suno-ai/bark.git pip uninstall torch torchaudio pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu117

In the python code I had:

import scipy import os import torch

os.environ["SUNO_OFFLOAD_CPU"] = "True" os.environ["SUNO_USE_SMALL_MODELS"] = "True"

from transformers import AutoProcessor, BarkModel

processor = AutoProcessor.from_pretrained("suno/bark") model = BarkModel.from_pretrained("suno/bark") model.to('cuda')

voice_preset = "v2/en_speaker_6"

inputs = processor("Text that you want to convert to speech.", voice_preset=voice_preset)

inputs.to('cuda') audio_array = model.generate(**inputs,pad_token_id=10500) audio_array = audio_array.cpu().numpy().squeeze() sample_rate = model.generation_config.sample_rate scipy.io.wavfile.write("bark_out.wav", rate=sample_rate, data=audio_array)

Nothing worked until I added the line above, i.e. inputs.to('cuda')

After that, I could see the GPU kicking in on Task Manager. I know diddly-squat about all this, but spotted the critical code in another post somewhere else. I may be taking extra steps that are not required, but hey ho - it's working now.

This one worked for my 4070 on Win11 using "normal" cmd.

suno-ai / bark

Unable to use GPU #342