Fix Stream Generator on MacOS

coqui-ai / TTS

🐸💬 - a deep learning toolkit for Text-to-Speech, battle-tested in research and production

http://coqui.ai

Mozilla Public License 2.0

31.64k stars 3.78k forks source link

Fix Stream Generator on MacOS #3792

Open gravityrail opened 2 weeks ago

gravityrail commented 2 weeks ago

Quick fix for streaming responses on macOS. It was crashing somewhere in pytorch. Unfortunately I can't find the original post where I found this patch to give credit - too many tabs open! But it works for me.

For this to work, you must run with PYTORCH_ENABLE_MPS_FALLBACK=1 and call model.to(mps_device) on the XttsModel

CLAassistant commented 2 weeks ago

All committers have signed the CLA.

eginhard commented 1 week ago

The crash occurs on any OS with transformers>=4.41. This fix was originally shared by @pseudotensor in https://github.com/idiap/coqui-ai-TTS/issues/31. But I just opened https://github.com/idiap/coqui-ai-TTS/pull/46 in our fork (since this repo is not updated anymore) which more generally aligns the streaming code to the upstream transformers implementation again, so it should be more resilient to future updates. It will be available in the next release.