search

HaiFengZeng / clari_wavenet_vocoder

Other
56 stars 17 forks source link

Need help to run clarinet #2

Open Thien223 opened 6 years ago

Thien223 commented 6 years ago

Thank you so much for your effort and sharing.

How can I train the clari_wavenet? By running train_student.py, isn't it? I have read your code, and I think running train.py is just original Wavenet, was I wrong?

You have 3 major models: wavenet, clari_wavenet and student_wavenet, but when reading your code, I realize that wavenet is teacher, clari_wavenet is student, and (not sure but i think) student_wavenet is student of parallel wavenet (not clari). Is this right?

zhf459 commented 6 years ago

Sorry for my bad.There are three models in this code, wavenet and parallel one and clarinet, train.py is used to train the wavenet with different output type like gaussian(mixture of gaussian), softmax, (mixture of logistic) and I modify some of code from original r9y9's code, use the gaussian output type and you may find the code of loss function DiscretizedGaussianLoss,so you can use train.py to train a teacher wavenet based on different output type as mentioned before(default use gaussian in clarinet).

So you can use the train.py to train clarinet(use --preset=./preset/ljspeech_gaussian.json).When the teacher is trained you can train a student network use train_student.py, you should provide the teacher model checkpoint (I trained single gaussian model about 500k, and sample sounds good)and use --preset=./preset/ljspeech_gaussian.json option .In train_student.py, build_model function can support different student model(clarinet or parallel).For now the training result is not very good, i will upload some samples if i get some good. some errors may exists in the code, hope you feel free to comment.

Thien223 commented 6 years ago

Thank for replying.

I'm working with Tensorflow Tacotron-2. My model synthesizes good results. Now I want to apply parallel Wavenet to speed up it.

I have to read, understand your code and try to deploy using Tensorflow, that should spend alot of efforts because I'm not a Tensorflow master. Anyway, thank you so much.

Thien223 commented 6 years ago

Sorry for bothering you, @zhf459 but Could you post some output samples of Clari (or Parallel) wavenet model, please?

I've checked other's results, but they are so bad. How is your model's ones?

Still trying to read and understand your code, it is unimaginable sophisticated for me. ^^. After listening to your output, I will try to figure out what made the out put be bad. and we can debug together.