Add Replicate demo and API

netease-youdao / EmotiVoice

EmotiVoice 😊: a Multi-Voice and Prompt-Controlled TTS Engine

Apache License 2.0

7.3k stars 624 forks source link

Add Replicate demo and API #55

Closed bramhooimeijer closed 10 months ago

bramhooimeijer commented 10 months ago

Hi!

Very interesting work on EmotiVoice. I saw in your roadmap that you are looking for a demo and API. This pull request:

Packages the model into Replicate's cog tool.
Makes a demo available on Replicate
Adds a reference to the demo on the README.md (only in english)
Exposes an API, via Replicate

To get the packaging to work, I have changed the relative paths of the model to the src (pull it out of the git filetree). Do have a look at that.

P.s. the speaker selection on the demo page seems to have a non-working default. If you select a name from the list it works. Will see if I can get that fixed.

If you want the replicate page to point at your repo that is of course preferred

syq163 commented 10 months ago

Great job! Thank you very much for doing that. We'll aim to merge it as soon as we can.

syq163 commented 10 months ago

@bramhooimeijer I'm merging the PR. Is it possible to avoid changing 'the relative paths of the model'? I'm concerned it could conflict with the current Docker setup and user habits.

bramhooimeijer commented 10 months ago

Hi @syq163, I will have a look at the paths. The origin is:

I am downloading the models in cog.yaml. These actions are executed before the repo is set up, so that is why I moved folders. I'll see whether I can move the model downloads to a later stage.
The paths are not really consistenly used throughout the repo. For example, config/joint/config.py specifies the style_encoder_path, but this path is not used in the demopage or am_encoder_inference.py. Perhaps it is good to harmonize this first?

What is your plan for the download location of the models? Some are now in LFS, other on drive? It is not easy to just wget the models from google drive.

bramhooimeijer commented 10 months ago

@syq163 ,

This should be fixed now. If you look at 'files changed', no original files are changed outside of the readme.

Currently, the predict.py uses the model from a cache on replicate. Latest revision is available on Replicate as well.

In the future, we could rework predict.py to rely on inference_am_vocoder_joint.py as well, to improve code reuse. Perhaps it is good to first harmonize demopage.py and inference_am_vocoder_joint.py.

For now, this can be merged from my point of view. Let me know if you have any more remarks.

syq163 commented 10 months ago

Thank you for taking care of that. I believe it is now in good shape, and I will merge this PR promptly.