Amphion VALL-E new version release

[jiaqili3]

✨ Description

In this PR, we release an unofficial PyTorch implementation of VALL-E, a zero-shot voice cloning model via neural codec language modeling. If trained properly, this model could match the performance specified in the original paper. This is a refined version compared to the first version of VALLE in Amphion, we have changed the underlying implementation to Llama to provide better model performance, faster training speed, and more readable codes. This can be a great tool for users who want to learn speech language models and its implementation.

🚧 Related Issues

None

👨‍💻 Changes Proposed

• • [x] We have changed the underlying implementation to Llama to provide better model performance, faster training speed, and more readable codes.
• • [x] We provide more detailed README.md for reproducing our models with pretrained weights, training on LibriTTS, and future plans on improving the model.
• • [x] We use a refined codec model name SpeechTokenizer as the codec, yielding better modeling quality than the original Encodec

🧑‍🤝‍🧑 Who Can Review?

@HeCheng0625 @RMSnow @HarryHe11 @zhizhengwu

✅ Checklist

• • [x] Code has been reviewed
• • [x] Code complies with the project's code standards and best practices
• • [x] Code has passed all tests
• • [x] Code does not affect the normal use of existing features
• • [x] Code has been commented properly
• • [x] Documentation has been updated (if applicable)
• • [x] Demo/checkpoint has been attached (if applicable)

[RMSnow]

Do we have any pretrained models or demo for this new valle?

[jiaqili3]

Do we have any pretrained models or demo for this new valle?

It has been detailed in the readme file in egs/tts/valle_v2, and the demo.ipynb has also been uploaded to run inference with pretrained weights

[jiaqili3]

Hi @RMSnow , thanks for your review! I've updated the code and your previous review questions have been resolved.

[RMSnow]

Hi @jiaqili3, please update the demo.ipynb. Others look good to me.

[jiaqili3]

Updated. Thanks @RMSnow