Improvement Ideas over Basic Deep Q-Network Control

[joon0503]

Current design is the most basic architecture for deep RL. Followings are some improvements which can be made for Q-learning.

• [x] Experience Replay
• [x] Usage of 'Targent Network' (See deepmind paper @ https://storage.googleapis.com/deepmind-media/dqn/DQNNaturePaper.pdf)
• [x] Prioritized Experience Replay (https://arxiv.org/pdf/1511.05952.pdf)
• [x] Dueling network
• [x] Double network (https://arxiv.org/pdf/1511.06581.pdf)
• [x] Using multiple frames + CNN
• [ ] Hindsight Experience Replay (HER) https://arxiv.org/pdf/1707.01495.pdf
• [ ] Using past inputs
• [ ] Error clipping

Incorporate Input Cost

• [ ] Using past inputs, penalize for using different input
• [ ] Modify current input. Instead of using steering angle, get current steering and outputs the whether to do nothing, steer more to left, or to right.

References https://github.com/Silvicek/DQN-for-autonomous-vehicles https://medium.freecodecamp.org/improvements-in-deep-q-learning-dueling-double-dqn-prioritized-experience-replay-and-fixed-58b130cc5682

[joon0503]

Implemented target network.

[joon0503]

Implemented dueling network. It seems double network is just target network?

[joon0503]

Implemented double network and PER

[joon0503]

Added HER