search

google-deepmind / dqn_zoo

DQN Zoo is a collection of reference implementations of reinforcement learning agents developed at DeepMind based on the Deep Q-Network (DQN) agent.
Apache License 2.0
457 stars 78 forks source link

Atari no frame skip, no sticky actions? #13

Closed slerman12 closed 3 years ago

slerman12 commented 3 years ago

I'm a bit confused. Aren't these procedures usually incorporated in Atari? Looking at gym_atari briefly, it seems like they're not incorporated here, correct me if I'm wrong. Also, I don't see where the image frame is down-sampled. In other libraries, like rlpyt, they reduce the size of the image in half (in addition to the other steps).

jqdm commented 3 years ago

No frame skipping: we chose to have the preprocessing on the side of the agent instead of in the environment, see the implementation notes in the README.md for the rationale.

No sticky actions: one of the main aims of these agent implementations is to be close as reasonably possible to the corresponding papers. All the papers did not train with sticky actions enabled so it was deliberately disabled here.

Downsampling: this is done here along with the other preprocessing, including frame skipping / action repeats.