search

semperai / amica

Amica is an open source interface for interactive communication with 3D characters with voice synthesis and speech recognition.
https://heyamica.com
MIT License
581 stars 92 forks source link

add RVC support for changing the voice from tts #71

Closed aziib closed 1 month ago

aziib commented 6 months ago

i like using xtts, but it's very slow compare to normal tts like ljspeech or speechT5. can you add RVC support where i can put my rvc model in any tts so the tts voice will change to my rvc model voice. similar like sillytavern rvc extension convert the output tts voice into rvc first and then show the message and play the output tts after the voice is changed. https://github.com/SillyTavern/SillyTavern-Extras https://github.com/RVC-Project/Retrieval-based-Voice-Conversion-WebUI

slowsynapse commented 4 months ago

Arbius has a $100 AIUS bounty for this issue!

Brief: Add RVC Support as outlined.

Please read carefully:

To begin work on a bounty, reply by saying “I claim this bounty” - you will have 48 hours to submit your PR before someone else may attempt to claim this bounty.

To complete the bounty, within 48 hours of claiming, reply with a link to your PR referencing this issue and an Ethereum address. You must comply with reviewers comments and have the PR merged to receive the bounty reward. Please be sure to focus on quality submissions to minimize the amount of time reviewers must take.

Eidenz commented 2 months ago

Hm, I got a few ideas to implement that. Won't take the bounty don't need it, I just want RVC as well and have the resources so I will have a look.

Eidenz commented 2 months ago

RVC has very poor API support, difficult to add to this project. ~~I'm thinking of using a fork of RVC with an integrated FastAPI https://github.com/SocAIty/Retrieval-based-Voice-Conversion-FastAPI~~

Will do my tests on this one from now on, as on top of providing a functioning API it allows sending binary directly, useful in our case.

Edit: investigating TTS-RVC-API (https://github.com/skshadan/TTS-RVC-API), which combines TTS with RVC instead of relying on two separate services. This would make RVC a full replacement TTS in Amica.

flukexp commented 2 months ago

I claim this bounty

#97

Eidenz commented 2 months ago

Yup good job, better than the TTS-RVC-API I was checking in the end as it lacks rmvpe, which is the best-sounding one with small GPU usage.

slowsynapse commented 1 month ago

Thanks for the help @Eidenz

slowsynapse commented 1 month ago

Done and bounty sent!