webaverse / app

Web metaverse client
https://app.webaverse.com
MIT License
344 stars 207 forks source link

Integrate new voice models #3678

Open avaer opened 2 years ago

avaer commented 2 years ago

We have a voice training pipeline now, with additional trained voices, so it's time to integrate them.

Our talknet implementation (https://github.com/webaverse/tiktalknet) has code for pulling all of the PPP pones models, but we already have them all on the sever.

Steps:

0) find the voice model files on the voice server 1) back up all of the model files from the server -> gdrive 2) update the code to only pull local models, instead of trying to download with a local cache. this code isn't even running anymore since everything is cached. 3) add the new voice models on the backend. for this you will need to make up some fake gdrive ids which are not actually gdrive ids, purely for the purpose of identifying the voice in the API. 4) add the voice models on the frontend 5) refactor the backend + frontend code to not use GDrive ids, but more natural voice keys for identification