search

weilinie / RelGAN

Implementation of RelGAN: Relational Generative Adversarial Networks for Text Generation
MIT License
120 stars 31 forks source link

Teacher Forcing during Adversarial Training #15

Closed abhishek18124 closed 4 years ago

abhishek18124 commented 4 years ago

From the code, I can see that teacher-forcing was used during the pretraining. However, during the adversarial training, it is not used. Was this decision based on any experiments? If yes, can you explain them?

weilinie commented 4 years ago

This is a good point. The initial motivations of using adversarial training is to avoid the so-called exposure bias in teacher forcing, and I guess this is why most text GANs didn’t use teacher forcing. I didn’t specifically compare adversarial training with or without teacher torching, but I think it’s worthy to try and see how the comparison results look like.