search

Farama-Foundation / Arcade-Learning-Environment

The Arcade Learning Environment (ALE) -- a platform for AI research.
GNU General Public License v2.0
2.12k stars 420 forks source link

For V5, frame skip is 5 by default, why? #443

Closed pseudo-rnd-thoughts closed 2 years ago

pseudo-rnd-thoughts commented 2 years ago

I'm implementing a wrapper of the V5 environment that includes frame skipping and stacking, etc Looking at the default constructor, frame skipping is set to 5. Why? Am I missing a paper that explains the advantage of 5 compared to 4 which to my knowledge was the default (https://arxiv.org/pdf/1312.5602.pdf)

Thanks

JesseFarebro commented 2 years ago

Hi @pseudo-rnd-thoughts,

A frame-skip of four was used in the original DQN paper. Note that the ALE pre-dates DQN and a frame-skip of five was common amongst prior works. In the latest methodology review (Revisiting the Arcade Learning Environment) a frameskip of 5 was used when generating the empirical results.

With that being said, we would recommend you use a frame-skip of 4 if you want to align with the prevailing consensus in the Deep-RL literature. The v5 default of 5 is due to a miscommunication on my part and the default will be changed to 4 in the next release to align with the post-DQN literature. Sorry, for the confusion.

Let me know if you have any other questions regarding the most recent methodology.

pseudo-rnd-thoughts commented 2 years ago

Hi Jesse,

That makes sense, thanks for the update

Mark