search

cmusphinx / g2p-seq2seq

G2P with Tensorflow
Other
667 stars 196 forks source link

Unable to replicate results #173

Closed robrechtme closed 5 years ago

robrechtme commented 5 years ago

When I run the metrics of the pretrained model, using the Phonetisaurus test split I get:

Which is what I expected.
However, when I train my own model with the training data provided with Phonetisaurus, I am not able to replicate the results:

I used the same configuration: 3 layers, 256 units and trained until early stopping at 200k steps. Why am I getting this big performance difference?

nshmyrev commented 5 years ago

It has been quite some time ago and I'm sorry I am afraid the proper config is lost. Also T2T has some regressions over time, they overoptimize for translation config but g2p might be worse. I would try the following: play with learning rate and batch size, don't rely on early stop, train for more iterations. I might check this, but it will not be fast.

Thank you for the report.

robrechtme commented 5 years ago

Thanks for your clarification.
According to Tensorboard, the models could indeed be trained a bit longer. I'll look into it if I have the time.

robrechtme commented 5 years ago

OK, after a lot of experimenting I finally managed to receive similar results:

nshmyrev commented 5 years ago

Cool, please provide us the details of your training.