jiegenghua / paper-reading

0 stars 0 forks source link

2016-A connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and energy based models-Pieter Abbeel #3

Open jiegenghua opened 5 years ago

jiegenghua commented 5 years ago

Generative adversarial networks(GAN): a class of generative models in which a generator is trained to optimize a cost function that is being simultaneously learned by a discriminator.

Learning the cost function underlying observed behavior is known as IRL. Certain IRL methods are in fact mathematically equivalent to GANs.

This paper demonstrates an equivalence between a sample-based algorithm for maximum entropy IRL and a GAN in which the generator's density can be evaluated and is provided as an additional input to the discriminator. maximum entropy IRL is a special case of an energy-based model. This paper highlights the connection between GANs, IRL, and EBM(energy-based model). image

image

image

image

image

image

image

GAN can be viewed as a sample-based algorithm for the MaxEnt IRL problem.

image

GAIL: combing GAN with RL.======MaxEnt IRL