[Proposal] Initial State Tracking - Bipedal Walker Hardcore

[arthur-plautz]

Proposal

Create methods that are able to retrieve and set, the initial states that uniquely represents the terrain created for Bipedal Walker Hardcore.

Motivation

I'm currently working on a research on curriculum learning, based on a paper that used the bipedal walker hardcore problem to demonstrate that a selection on the initial conditions of the experiment could improve the overall performance. My current work implies extracting the states used to create the terrain, so that I can use it to automatically select this initial states along the experiment, and also set the states to generate a specific terrain. I was working on some changes in the bipedal environment locally, but it occurred to me that this could benefit other people that may have the same issue.

Pitch

The idea is to have a parameter in the reset method that allows the overwriting of the states used to generate the terrain, and also the persistence of those states, available through an attribute, similar to the terrain attribute.

Alternatives

We may have other ideas of how to do it, but I think the solution would involve a method to set the states and an attribute to access them. Feel free to make your suggestions 😄

Additional context

The paper: Automated curriculum learning for embodied agents a neuroevolutionary approach. Current Fork: https://github.com/arthur-plautz/gym

Checklist

• [X] I have checked that there is no similar issue in the repo

[pseudo-rnd-thoughts]

This should be possible with the options argument to pass an intended state for the environment to use

[arthur-plautz]

Yes, but the options argument is not being used anywhere on the reset() method: https://github.com/Farama-Foundation/Gymnasium/blob/main/gymnasium/envs/box2d/bipedal_walker.py

It's possible that it was supposed to be included on the super().reset() method? Sorry if I missed something here.

[pseudo-rnd-thoughts]

What I mean is that this solution could be implemented using the options arguments such that the initial state can be passed through options, i.e.,options["state"] = new_state.

[arthur-plautz]

Oh, I see, I'll make the changes then open a PR, thank you for your help!