Closed AranKomat closed 6 years ago
Thanks for your clarification and in particular telling my that LSTM worked not significantly better than CNN.
Sorry, I mean that LSTM limited by the speed because of its bad parallelization. While the performance of LSTM depends on the specific task.
I see. It'd be interesting to try with Transformer. Sorry but I have three questions remaining.
In SeqGAN issue page, LantaoYu theoretically justified training G once for each 15 steps of D updates. But did you guys empirically verify that the 1:15 ratio was good?
In the adversarial training, it was trained for 800 batches only. Is this small number due to fast convergence or considerable amount of training time?
Does the discriminator of LeakGAN receive unfinished real sentences as well, to discriminate them from unfinished fake sentences? If all the discriminator receives are real finished sentences and fake finished and unfinished sentences, then it's trivial to discriminate them by judging the one with comma at the end to be real one. But in this way, the discriminator cannot learn much. How is this avoided in your algorithm?
There are some recent text generation GAN papers in which the discriminator is a LSTM rather than a CNN. Why was CNN used in this paper?