Add PicoAudio Model - Githubissues

open-mmlab / Amphion

Amphion (/æmˈfaɪən/) is a toolkit for Audio, Music, and Speech Generation. Its purpose is to support reproducible research and help junior researchers and engineers get started in the field of audio, music, and speech generation research and development.

https://openhlt.github.io/amphion/

MIT License

4.45k stars 379 forks source link

Add PicoAudio Model #249

Open zeyuxie29 opened 1 month ago

zeyuxie29 commented 1 month ago

✨ Description

The PR adds the PicoAudio into the Amphion toolkit.

PicoAudio: Enabling Precise Timestamp and Frequency Controllability of Audio Events in Text-to-audio Generation

repo: https://github.com/zeyuxie29/PicoAudio
paper: https://arxiv.org/abs/2407.02869v2
demo: https://zeyuxie29.github.io/PicoAudio.github.io/
huggingface spcae: https://huggingface.co/spaces/amphion/PicoAudio

🚧 Related Issues

[List the issue numbers related to this PR]

👨‍💻 Changes Proposed

[x] Added the dataloader and model implement of PicoAudio into models/temporally_controllable_tta
[x] Added the training and inference scripts of PicoAudio into models/temporally_controllable_tta

🧑‍🤝‍🧑 Who Can Review?

@zhizhengwu @HeCheng0625

🛠 TODO

✅ Checklist

[ ] Code has been reviewed
[ ] Code complies with the project's code standards and best practices
[ ] Code has passed all tests
[ ] Code does not affect the normal use of existing features
[ ] Code has been commented properly
[ ] Documentation has been updated (if applicable)
[ ] Demo/checkpoint has been attached (if applicable)