search

neonbjb / tortoise-tts

A multi-voice TTS system trained with an emphasis on quality
Apache License 2.0
13.23k stars 1.83k forks source link

is M1 support a proof of concept for now? #773

Open working-name opened 6 months ago

working-name commented 6 months ago

Hi there, got a 24gb ram m2 mac and python insists on using 27gb of RAM. Obviously... problematic. I ran this:

python tortoise/do_tts.py --text "I'm going to speak this" --voice random --preset fast

Took a good 5 minutes, almost crashed the OS.

tortoise/read_fast.py demands CUDA on macOS on an M2. 🤷

This also abuses the crap out of my machine, but the progress bar doesn't move at all, stays at 0% after 2-3 minutes. 

PYTORCH_ENABLE_MPS_FALLBACK=1 python tortoise/read.py --textfile test.txt --voice emma

So the question remains: is it a proof of concept? Are we doomed if we don't buy NVIDIA crap?

btesanovic commented 1 month ago

I have 16gb m2 and python is using 19+gb and it constantly crashes with MPS out of memory error

Looking at the code it seems that culprit is in api.py method pick_best_batch_size_for_gpu() which always returns 16 what I did was just retuned 8 as a batch size on line 195

self.autoregressive_batch_size = pick_best_batch_size_for_gpu() if autoregressive_batch_size is None else autoregressive_batch_size
#just hard code if batch_size >16 to 8
if self.autoregressive_batch_size >15:
    self.autoregressive_batch_size=8