Variational Inference MPC for Bayesian Model-based Reinforcement Learning By: Masashi Okada, Tadahiro Taniguchi

[QiXuanWang]

Link: Arxiv

Problem:

Probabilistic ensembles with trajectory sampling (PETS) is a leading type of MBRL, which employs Bayesian inference to dynamics modeling and model predictive control (MPC)with stochastic optimization via the cross entropy method (CEM). we propose a novel extension to the uncertainty-aware MBRL

Innovation/Contribution:

Firstly, we introduce a variational inference MPC (VI-MPC),which reformulates various stochastic methods, including CEM, in a Bayesianfashion. Secondly, we propose a novel instance of the framework, called prob-abilistic action ensembles with trajectory sampling (PaETS).

Conclusion:

In comparison to PETS, our method consistently improvesasymptotic performance on several challenging locomotion tasks

Comments: This paper is published after "Benchmarking Model-Based Reinforcement Learning" link where author got the conclusion that MBRL is sub-optimal compared with MFRL. But will this paper proves that MBRL indeed is good enough?

[QiXuanWang]

I think this paper is an extension to PETS algorithm.

[QiXuanWang]

Reference Paper: Variational Inference: A Review for Statisticians link Reference Video to Variational Inference https://www.youtube.com/watch?v=ogdv_6dbvVQ