Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio
2. Tags
GAN
Generative Model
2-Player-Game
Computer Vision
3. Summary
Deep learning has experienced great accomplishments in past years, but most of it belonged to discriminative models that used back propagation and dropout algorithms with huge success.
Research on generative models were mostly accompanied by approximation inference or Markov chains.
With two separate models - G and D - competing against each other, we can train a generative model only using back propagation, dropout and forward propagation, which all had been proved successful.
GANs share some commonalities with VAEs. They are different in that
(1) the second neural network in VAEs perform approximation inference and
(2) GANs cannot model discrete data while VAEs cannot have discrete latent variables.
In GAN, generator tries to learn the probability distribution of original data starting from input noises,, while the discriminator tries its best to tell original data from samples generated by G.
Value function for the two player minmax game:
The process of optimization looks like following:
(a) is the initial state. In (b), D optimizes and converges to D*(x) = P_{data}(x) / (P_{data}(x)+p_{g}(x))
In (c), G is updated, and gradient of D has guided G to move to regions that are more likely to be judged as data. (Gets better at faking)
After numerous steps of training, the model reaches a global optimum where samples generated by G is virtually identical to the original data and D can no more tell the difference.
Details of the training algorithm is written below:
Two theorems are introduced.
(1) p_g = p_data is the one and only global minimum of the training criterion.
(2) p_g converges to p_data under some assumptions. (Enough capacity, improving updates...)
There are advantages and disadvantages for this models.
Advantages: No approximation inference / Markov chain. Freedom in function choice.
Disadvantages: No explicit representation of p_g(x). D must keep pace well with G in training because otherwise G might just collapse.
Comparison between generative models.
Summary on the model framework.
4. Personal Comments
I've heard about this model almost ever since I set my foot on the field, but now that I've read it myself, I rediscover how creative this idea has been.
Among many topics of ML/DL, generative models always astonish me the most in that it outputs something new; something creative that only humans were believed capable of generating.
Since I've read the paper on Cycle-GAN first, it strikes me how much work has been made on top of this powerful idea. Restyling on Impressionist artists were amazing.
I can't wait to make codes of this model myself and experience the 'generative force' of DL.
Still the most powerful techniques in ML is back propagation and dropout.
1. Authors
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio
2. Tags
GAN
Generative Model
2-Player-Game
Computer Vision
3. Summary
Deep learning has experienced great accomplishments in past years, but most of it belonged to discriminative models that used back propagation and dropout algorithms with huge success.
Research on generative models were mostly accompanied by approximation inference or Markov chains.
With two separate models - G and D - competing against each other, we can train a generative model only using back propagation, dropout and forward propagation, which all had been proved successful.
GANs share some commonalities with VAEs. They are different in that
In GAN, generator tries to learn the probability distribution of original data starting from input noises,, while the discriminator tries its best to tell original data from samples generated by G.
Value function for the two player minmax game:
The process of optimization looks like following:
Two theorems are introduced.
There are advantages and disadvantages for this models.
Comparison between generative models.
Summary on the model framework.
4. Personal Comments