[Question] DQN vs Open AI Baseline's Rainbow agent

hill-a / stable-baselines

A fork of OpenAI Baselines, implementations of reinforcement learning algorithms

http://stable-baselines.readthedocs.io/

MIT License

4.16k stars 725 forks source link

[Question] DQN vs Open AI Baseline's Rainbow agent #77

Closed wilkinsmicawber closed 5 years ago

wilkinsmicawber commented 5 years ago

The rainbow agent by default experienced the best base result in sonic for the OpenAI team by a large margin, if you exclude the ridiculously resource intensive parallel PPO training:

https://arxiv.org/pdf/1804.03720.pdf

Is the DQN agent provided by stable-baselines the rainbow model?

araffin commented 5 years ago

Hello,

Is the DQN agent provided by stable-baselines the rainbow model?

The DQN agent is not the rainbow model, however it integrates:

dueling DQN
prioritized experience replay
parameter noise (from OpenAI)

So the following are missing:

n-step Bellman updates
distributional RL

If you want an implementation of Rainbow, I recommend you to check Dopamine repo ;)

https://github.com/google/dopamine/blob/master/dopamine/agents/rainbow/rainbow_agent.py