CPU only?

[jondana]

It seems to be running in CPU only, even though I used this prompt to start it in Docker:

docker run -it --rm --gpus all -p 7860:7860 athomasson2/ebook2audiobookpiper-tts:latest

Am I doing something wrong? Looks like it is possible based on this thread: https://github.com/rhasspy/wyoming-piper/pull/5

Thanks!

ps. While I found that xtts has better phrasing, the hallucinations (made up words, vocal artifacts) are worse than Piper which make the narration harder to understand unfortunately. I tried messing with a lot of the advanced settings (temperature etc. but couldn't get xtts more consistent than the base settings).

I am excited for Piper finetuning btw! Is there a way to be notified when finetuning is available? Also, I have made some high quality xxts finetunes, I would be happy to send them to you to attach to the project if you want. Just let me know!

[DrewThomasson]

About the piper-tts GPU use

• hm, not gona lie, I just never looked at this repo in a while XD, it's defiantly possible to have gpu work
• I'll look at adding that --cuda thing when I find time :) 👍

About fine-tuning piper tts

you should be able to fine tune pipertts , although its much harder to lol

Docs on it

https://github.com/rhasspy/piper/blob/master/TRAINING.md

Youtube Video on it

https://www.youtube.com/watch?v=67QvWOp3dkU

You say Hallucinations are the issue?
I might have a better repo that doesn't hallucinate lol
(Only limitation is that styletts2 only works for English)
ebook2audiobookStyletts2

https://github.com/DrewThomasson/ebook2audiobookSTYLETTS2

• It's pretty close to xtts with easy voice cloning, just less emotional, but runs a lot faster
• I forgot if I gave it GPU functionality tho lol but it should auto-detect gpu

[DrewThomasson]

ALSO You say you have some high quality xtts fine-tunes? 👀

Slap em on hugginface, send me links to them I'll check them out ! 😄

• I'm planning on eventually getting a database of known xtts fine-tuned models to add to the ebook2audiobookxtts project 😄

[DrewThomasson]

I can confirm the fine-tuning piper-tts colabs still work as of this moment lol

I've used them before to fine tune a piper-tts model on Death from puss and boots 😅 https://huggingface.co/drewThomasson/piper_tts_finetune_death_from_puss_and_boots

Be aware it's very labor intensive for fine-tuning piper-tts 😭

• They made XTTS fine-tuning so much easier 😢

[jondana]

Perfect! Here is the first one. I'll get the rest to you later: https://huggingface.co/jdana/Bob_Newfound

Styletts2 was worse than Piper and xtts from my very limited experiments I just did

I will have to try out that piper finetune tutorial, thanks for sharing! With the tech moving so fast, I wonder if it is worth it to dive into that now or just wait until someone makes an easy webui 😂

[jondana]

Wait how do I find the link to the Piper-tts Colab?

[DrewThomasson]

Noted lol

I'm uploading the newest docker image for piper tts that supports GPU! :)

school wifi is slow 😅

[DrewThomasson]

Wait how do I find the link to the Piper-tts Colab?

The piper-tts Notebooks should be found here? 🤔 https://github.com/rhasspy/piper/tree/master/notebooks

[jondana]

Noted lol

I'm uploading the newest docker image for piper tts that supports GPU! :)

school wifi is slow 😅

Hell yeah! Thanks so much, and that was crazy fast, I hope you are prioritizing your studies btw 😟

Wait how do I find the link to the Piper-tts Colab?

The piper-tts Notebooks should be found here? 🤔 https://github.com/rhasspy/piper/tree/master/notebooks

I thought you meant that there was a piper training workflow in a Google Colab 😅 I got excited lol

[DrewThomasson]

Yeah the notebooks are for opening in google colab

If you click on any of them you should see a button for "open in google colab" lol

[DrewThomasson]

Here cause school wifi is too slow

DOCKER PIPER-TTS GPU

docker run -it --gpus all -p 7860:7860 --platform=linux/amd64 registry.hf.space/drewthomasson-ebook2audiobookpiper-tts-gpu:latest python app.py

You can also duplicate the hugginface space if you want to give it a gpu from huggingface at like 40 cents an hour lol

https://huggingface.co/spaces/drewThomasson/ebook2audiobookpiper-tts-GPU

[jondana]

Yeah notebooks for google colab lol, If you click on any of them you should see a button for "open in google colab" lol

Woah! Great, thank you!

Here cause school wifi is too slow

DOCKER PIPER-TTS GPU

docker run -it --gpus all -p 7860:7860 --platform=linux/amd64 registry.hf.space/drewthomasson-ebook2audiobookpiper-tts-gpu:latest python app.py

You can also duplicate the hugginface space if you want to give it a gpu from huggingface at like 40 cents an hour lol

https://huggingface.co/spaces/drewThomasson/ebook2audiobookpiper-tts-GPU

🔥🔥🔥🔥🔥🔥🔥🔥🔥

[jondana]

Hmm, still having trouble getting Piper to work with GPU

Specified provider 'CUDAExecutionProvider' is not in available provider names.Available providers: 'AzureExecutionProvider, CPUExecutionProvider'
  warnings.warn(
Model loaded successfully.

[DrewThomasson]

Do you have a Cuda capable Nvida GPU? cause it only works with Nvidia GPU's

What output do you get when you run this?

docker run --gpus all nvidia/cuda:11.0-base nvidia-smi

[DrewThomasson]

Here, I tried fixing it by adding

nvidia-cuda-toolkit
libcudnn8
libcudnn8-dev
libegl1
libopengl0
libxcb-cursor0

into the packages.txt for the huggingface space

Try again with this comand to make it force pull the latest docker image and see if this works ✨

docker run -it --gpus all -p 7860:7860 --platform=linux/amd64 --pull always registry.hf.space/drewthomasson-ebook2audiobookpiper-tts-gpu:latest python app.py

[DrewThomasson]

I also created a free google colab of it with GPU support! :)

• Confirmed to work in my testing
• Pretty fast, aorund 1-2 seconds per sentence running on GPU :)

[jondana]

Hm, I tried the new pull and still not working, also the Colab says notebook not found

I am on Windows 10, RTX 3090

docker run --gpus all nvidia/cuda:12.3.1-base-ubuntu22.04 nvidia-smi The prompt above runs, not this one that you sent me: docker run --gpus all nvidia/cuda:11.0-base nvidia-smi

Does it have to be v11?

If it is confirmed to work on Colab it must be something on my end that I can troubleshoot.

[DrewThomasson]

OP you right, here try this for the google colab now I gave the wrong link

https://colab.research.google.com/github/DrewThomasson/ebook2audiobookpiper-tts/blob/main/notebooks/ebook2audiobook_piper_tts.ipynb"

Or this:

[jondana]

Were you able to confirm in the Colab that it was using the GPU, not the CPU?

I don't have Colab Pro so I can't check

[DrewThomasson]

Yes I can confirm that the Google colab runs on GPU and gives you FREE GPUs

You don't need Colab Pro to use the GPUS of google colab

You can see here on the free version

[jondana]

lol I do but I wanted to check when the program was running if it was utilizing the CPU or the GPU, like task manager etc.

Pretty sure you need to do something like this to see live if the GPU is being used?

[jondana]

The updated pull runs fine on my computer btw, not sure that wasn't clear. But it still uses the CPU instead of my 3090

[DrewThomasson]

wait I think your right >:|

I'll try another method and see what happens

[DrewThomasson]

Welp finally got it to work,

I'll try swapping out a command or something

[DrewThomasson]

I got that result running this test script in google colab

!pip install piper-tts
!pip install onnxruntime-gpu

import subprocess
import time

start_time = time.time()

subprocess.run(
    "echo 'Welcome to the world of speech synthesis I am a bot!Welcome to the world of speech synthesis I am a bot!Welcome to the world of speech synthesis I am a bot!Welcome to the world of speech synthesis I am a bot!Welcome to the world of speech synthesis I am a bot!Welcome to the world of speech synthesis I am a bot!Welcome to the world of speech synthesis I am a bot!Welcome to the world of speech synthesis I am a bot!Welcome to the world of speech synthesis I am a bot!Welcome to the world of speech synthesis I am a bot!Welcome to the world of speech synthesis I am a bot!Welcome to the world of speech synthesis I am a bot!' | piper --model en_US-lessac-medium --output_file welcome.wav --cuda True",
    shell=True
)

end_time = time.time()
duration = end_time - start_time
print(f"GPU CUDA Command took {duration:.2f} seconds to run.")

import subprocess
import time

start_time = time.time()

subprocess.run(
    "echo 'Welcome to the world of speech synthesis I am a bot!Welcome to the world of speech synthesis I am a bot!Welcome to the world of speech synthesis I am a bot!Welcome to the world of speech synthesis I am a bot!Welcome to the world of speech synthesis I am a bot!Welcome to the world of speech synthesis I am a bot!Welcome to the world of speech synthesis I am a bot!Welcome to the world of speech synthesis I am a bot!Welcome to the world of speech synthesis I am a bot!Welcome to the world of speech synthesis I am a bot!Welcome to the world of speech synthesis I am a bot!Welcome to the world of speech synthesis I am a bot!' | piper --model en_US-lessac-medium --output_file welcome.wav",
    shell=True
)

end_time = time.time()
duration = end_time - start_time
print(f"CPU Command took {duration:.2f} seconds to run.")

[DrewThomasson]

Nevermind it appears to be broken lol

The files produced by the GPU were actually empty

So >:|

[DrewThomasson]

I'm putting this on hold,

As it doesn't appear to be possible

Hit me up if you get piper itself running on your computer tho utilizing the GPU

Cause I can't figure it out for the life of me

Wouldn't even have to be modifying my code all I'd need is some example of any line of code that runs on GPU :(

[jondana]

lol thanks for trying, I will give it a go and see if I can make any headway. CPU speeds are workable, I am just curious to see how much faster it is using GPU 😄

[DrewThomasson]

Same lol

I can't Seem to find any definitive answer of it online for how much faster it is :/

[DrewThomasson]

Well update: i found this space

looks like it might be able to use the gpu?

https://huggingface.co/spaces/Gregniuki/Pipertts

When I Duplicated the space on gpu it seemed to work for me hmmm but GPU didn't seem that much faster . . . which is odd

you could attempt at running it locally as a docker perhaps?

here is my gpu duplicate version that should work on your computer as a docker https://huggingface.co/spaces/drewThomasson/Pipertts

docker run -it -p 7860:7860 --platform=linux/amd64 \
    -e HF_TOKEN="YOUR_VALUE_HERE" \
    registry.hf.space/drewthomasson-pipertts:latest 

[DrewThomasson]

Do tell if you see it utilizing your GPU tho if you test it out :)

[jondana]

Well update: i found this space

looks like it might be able to use the gpu?

https://huggingface.co/spaces/Gregniuki/Pipertts

When I Duplicated the space on gpu it seemed to work for me hmmm but GPU didn't seem that much faster . . . which is odd

you could attempt at running it locally as a docker perhaps?

here is my gpu duplicate version that should work on your computer as a docker https://huggingface.co/spaces/drewThomasson/Pipertts

docker run -it -p 7860:7860 --platform=linux/amd64 \
  -e HF_TOKEN="YOUR_VALUE_HERE" \
  registry.hf.space/drewthomasson-pipertts:latest 

Hm, can't get the webui to open

I tried pretty hard to change some files around with the original ebook2audiobook package but kept running into Docker build problems when installing 😑

Maybe I will try a little more, I feel like I am close.

Would be interesting if it ends up that the CPU version is faster for some reason 😶

[DrewThomasson]

Yeah doesn't seem like the space utilizes the GPU Nevermind

:/

[jondana]

I tried a bunch of stuff and couldn't get it to work.. sigh.. ur guys' job is hard

I will move on to trying out Piper finetunes!

[DrewThomasson]

https://huggingface.co/spaces/Gregniuki/Pipertts/discussions/1#672c0972b28c400174cb6e43

Ok so getting somewhere 👀

[DrewThomasson]

Ok SO WE HAVE An ANSWER ON SPEED INCREASE

From the guy:

On 1050 to was 5x speed of hugging spaces CPU especially for long 10 minutes and longer texts

[jondana]

Nice! I tried adapting his app code, changing the onnxruntime-gpu version, torch etc. for my 3090 to no avail. lol it feels so doable but I haven't had any luck. Curious to see what kind of performance he was getting in his experiments. Thanks for keeping the dream alive

[jondana]

Oh just saw your update! That's a massive increase! Dang that is so tantalizing especially because my card is so much faster than the 1050