search

facebookresearch / fairseq

Facebook AI Research Sequence-to-Sequence Toolkit written in Python.
MIT License
30.22k stars 6.38k forks source link

Textless NLP: missing tacotron2 training script for GSLM #3951

Open ahazeemi opened 2 years ago

ahazeemi commented 2 years ago

🐛 Bug

There is no script provided to train the tacatron2 in the following path: fairseq/examples/textless_nlp/gslm/unit2speech/tacotron2

Can we please get the training script as well so that we are able to train our own decoder (modified tacotron2) on a new dataset?

Thanks!

ahazeemi commented 2 years ago

@hikushalhere Can you please help us here?

ahazeemi commented 2 years ago

@eugene-kharitonov could you please help with this? Thank you!

ahazeemi commented 2 years ago

Can someone please help with this? Thanks

hikushalhere commented 2 years ago

As far as I remember, the training code for modified Tacotron2 was not open sourced with GSLM. @wnhsu, can you help?

ahazeemi commented 2 years ago

As far as I remember, the training code for modified Tacotron2 was not open sourced with GSLM. @wnhsu, can you help?

Thank you @hikushalhere for the update. Looking forward to @wnhsu's input on this.

wnhsu commented 2 years ago

@ahazeemi you can refer to the fairseq unit-to-speech example here, or the NVIDIA's Tacotron2 repo link.

ahazeemi commented 2 years ago

@ahazeemi you can refer to the fairseq unit-to-speech example here, or the NVIDIA's Tacotron2 repo link.

Thank you for the reply. After training the Transformer/FastSpeech2 model on units (as described here), can that be used in place of the modified tacotron in the GSLM speech resynthesis?

wnhsu commented 2 years ago

I don't think you can replace that since it's a reimplementation