MaskGAN: Better Text Generation via Filling in the ______

[howardyclo]

Metadata

• Authors: William Fedus, Ian Goodfellow and Andrew M. Dai
• Organization: Google Brain
• Conference: ICLR 2018
• Paper: https://arxiv.org/pdf/1801.07736.pdf
• Code: https://github.com/tensorflow/models/tree/master/research/maskgan

[howardyclo]

Summary

Neural text generation models are typically auto-regressive, trained via maximizing likelihood or perplexity and evaluate with validation perplexity. They claim such training and evaluation could result in poor sample quality (during sample generation, the model is often forced to condition on sequences that were never conditioned on at training time, leading to unpredictable dynamics in the hidden state of the RNN). Thus, they purpose to improve sample quality using actor-critic conditional GANs.

Related Work

• Professor Forcing: Making the hidden state dynamics to become predictable.
• Scheduled Sampling: Randomly conditioning on sampled words at training time.
• Both work indirectly to improve sample quality but no cost function to encourage high sample quality (this paper does so).