Closed hh0rva1h closed 2 years ago
Problem is more difficult on second look, I guess the registration code is right the way it is, but would wish a way to correct the max_episode_steps
when frameskip
is set to a different value when doing gym.make()
.
Hi @hh0rva1h, I agree the way we currently set max_episode_steps
isn't ideal as it doesn't dynamically update based on the frameskip value. The issue is that for legacy reasons we're using the default Gym wrapper to truncate the episode and it truncates based on the number of steps. Furthermore, you can only set this parameter when registering the environment so if you pass a different frameskip value when calling gym.make
there's no good way to update the wrapper.
In Gym we no longer use the TimeLimit
wrapper and specify the max number of frames ourselves. This solves any confusion with steps versus frames.
https://github.com/mgbellemare/Arcade-Learning-Environment/blob/969857fc0dd2cde24b425dba950aec73dbc89109/src/python/gym.py#L184 I think instead of
108000 // 4
it should read something like108000 // frameskip
here, shouldn't it? (From the quick first look I'm not sure how you would want to access the frameskip setting though at that point.)