AttA2C - Attention-based Curiosity-driven Exploration in Deep Reinforcement Learning
Author: Patrik Reizinger, MSc student in Electrical Engineering
Supervisor and Co-author: Márton Szemenyei, lecturer
Organization: Budapest University of Technology and Economics, Department of Control Engineering and Information Technology
Supplementary material for the paper Attention-based Curiosity-driven Exploration in Deep Reinforcement Learning submitted to ICASSP 2020. Preprint available at https://arxiv.org/abs/1910.10840.
Table of contents
General
The aim of the project is to develop new exploration strategies for Reinforcement Learning for agents which can generalize better. The focus is on curiosity-based methods, such as the Intrinsic Curiosity Module of the paper Curiosity-driven Exploration by Self-supervised Prediction, which is used extensively to build upon.
This project is implemented in PyTorch, using the stable-baselines package for benchmarking.
Proposed methods
- AttA2C (Attention-aided A2C): this new version of A2C utilizes attention to split the features fed into the Actor and the Critic.
- Action- and state-selective ICM: the extension of ICM aims to use attention for selectively use the features and actions in the forward and inverse dynamic models.
- Rational Curiosity Module (RCM): this novel curiosity formulation aim to incentivize the agent to exploit curiosity only if it contributes to generalization.
Results
Experiments were carried out on three Atari games: Breakout, Pong and Seaquest (v0 and v4 variants, the former is stochastic, as it uses action repeat with p=0.25).
Breakout
Pong
Seaquest
Cite
If you found this work useful, please cite the following paper:
@article{reizinger2019attention,
title={Attention-based Curiosity-driven Exploration in Deep Reinforcement Learning},
author={Reizinger, Patrik and Szemenyei, M{\'a}rton},
journal={arXiv preprint arXiv:1910.10840},
year={2019}
}