search

Farama-Foundation / Arcade-Learning-Environment

The Arcade Learning Environment (ALE) -- a platform for AI research.
GNU General Public License v2.0
2.14k stars 420 forks source link

Frame post-processing for reproducing the nature DQN. #439

Closed alirahkay closed 2 years ago

alirahkay commented 2 years ago

What is your suggestion for doing frame-postprocessing (taking the max over two grayscaled frames) similar to what Dopamine is doing? The solution in their repo would impose setting the frame-skip equal to one in the v5 version of the ale and then implementing the frame-skip manually! Is this the best practice?

JesseFarebro commented 2 years ago

Hi @alirahkay,

Yes, I'd suggest doing post-processing exactly how Dopamine is doing so. I've thought about moving frame maxing into the ALE itself (we have support for averaging successive frames) but maxing introduces other issues. Ideally, we would update the colour averaging algorithm to replace frame maxing but we haven't gotten around to this (#101).

So as you said, if you're using v5 environments you should create them via,

env = gym.make('ALE/Breakout-v5', frameskip=1)
env = AtariPreprocessing(env)

In the future, the Gym Atari wrapper might be factored out of Gym and at that point, it might be possible to make this easier for Gym users.