search

gmltmd789 / UnitSpeech

An official implementation of "UnitSpeech: Speaker-adaptive Speech Synthesis with Untranscribed Data"
https://unitspeech.github.io/
Other
133 stars 12 forks source link

Can I fine tune Text Encoder in other languages ? #4

Closed rishikksh20 closed 1 year ago

rishikksh20 commented 1 year ago

@gmltmd789 I have one question As Decoder working great to adapt on non english speaker, For training TTS for non-english language is it possible to just train pretrained Text Encoder on Lgrad and Lenc by freezing Decoder part ? I don't think I have to train Unit Encoder as it already trained on mHuBERT.

gmltmd789 commented 1 year ago

Hello,

Regarding your inquiry, we are currently conducting experiments internally.

Once these experiments are completed, we plan to release the code that allows for multi-lingual expansion, along with the training code for the model.

Thank you for your interest.

rishikksh20 commented 1 year ago

I am also working to finetune Text Encoder and MAS duration predictor for different languages.