jhuang448 / LyricsAlignment-MTL

MIT License
54 stars 12 forks source link

How to align Chinese songs? #3

Open IHikari opened 1 week ago

IHikari commented 1 week ago

Thank you for your work. I want to use this model for Chinese songs. What should I do? Do you know of any open-source Chinese lyrics alignment datasets? Your assistance and guidance would be greatly appreciated as it will significantly contribute to my project. Thank you once again for your invaluable support and dedication.

jhuang448 commented 1 week ago

Thank you for your interest in our work!

If you want to use the models in this repo, you need to convert Chinese characters to Latin strings (for example, pinyin), following jamendo's annotation style https://github.com/f90/jamendolyrics/tree/master/lyrics. Please follow the notebook example.ipynb.

These models are not trained on Chinese songs, so the performance might not be optimal. I don't know any open-source Chinese datasets which are large enough for lyrics alignment training, so I think using models trained on English data is a reasonable choice.

Please let me know how it works on Chinese if you decide to use it. I am also happy to answer further questions. Good luck with your experiments!