Eclectic-Sheep / sheeprl

Distributed Reinforcement Learning accelerated by Lightning Fabric
https://eclecticsheep.ai
Apache License 2.0
300 stars 29 forks source link

Why is Frame_skip for gymnasium mujoco environments set to zero? #94

Closed HiddeLekanne closed 11 months ago

HiddeLekanne commented 11 months ago

The default for these environments differs from environment to environment. If you set them all to zero, results become silently incomparable with other benchmarks which use the default values.

Also action_repeat = X is not necessarily equal to frame_skip = X. Mujoco Implementations use fixed step ODEs to solve their environments, meaning larger frame_skips result in larger (less accurate) steps.

belerico commented 11 months ago

Hi @HiddeLekanne, it was a leftover from a previous implementation: thanks for spotting out!
Regarding the frame_skip vs action_repeat:

HiddeLekanne commented 11 months ago

Ah I was just worried it was a design decision, and that the idea was to replace any frame_skip parameter with the action_repeat parameter. I am not doing any dmc anyways so that issue isn't on my radar.

Personally I am doing something with irregular timesteps and have added my own config args for that.

belerico commented 11 months ago

Beautiful! I'll have it removed asap Thanks