search

QiXuanWang / LearningFromTheBest

This project is to list the best books, courses, tutorial, methods on learning certain knowledge
9 stars 1 forks source link

Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review By: Sergey Levine #8

Open QiXuanWang opened 4 years ago

QiXuanWang commented 4 years ago

Link: Arxiv

This is one of the important paper that link MBRL with Variational Inference, published in 2018/05

Problem:

the connection between reinforcement learning and inference in probabilistic models is not immediately obvious

Innovation:

Discuss how a generalization of the reinforcement learning or optimal control problem is equivalent to exact probabilistic inference in the case of deterministic dynamics, and variational inference in the case of stochastic dynamics

Conclusion/Application:

  1. intersection of maximum entropy reinforcement learning and latent variable models
  2. design of reward functions

Comment:

  1. This paper put some theory background to connect RL with Statistical Inference. It first bring up the PGM (Probability Graph Model). Then in "Policy Search as Probabilistic Inference" section, Q/V value is defined by analogously to inference in HMM-style dynamic Bayesian networks
QiXuanWang commented 4 years ago

Supplemental Material: https://arxiv.org/ftp/arxiv/papers/1801/1801.07710.pdf