search

openai / gym

A toolkit for developing and comparing reinforcement learning algorithms.
https://www.gymlibrary.dev
Other
34.76k stars 8.61k forks source link

[Question] How to define an observation space that contains indeterminate numbers of array? #3154

Open wsdd2 opened 1 year ago

wsdd2 commented 1 year ago

Hi, I am trying to define a observation space in my custom env that, I have an agent flying in a 3-d space, and there are serveral obstacles in the space, and the agent cannot observe all obstacles in the space because its sight has a maximum distance limit. In other words, its observation contains two parts, the first part is its coordinate like [x, y, z], and the second part is the obstacles' coordinates in its sight (so its sight is like a ball with Radius is its max distance of sight) which will contains serveral indeterminate numbers of array like [x1,y1,z1] , [x2, y2, z2] ... However, the only information I could provide is the counts of obstacles in the whole space ( say 500), but I don't know how many obstacles will there be when it makes an observation. The question is, based on the environment described above, how should I define its observation space using gym.spaces?

pseudo-rnd-thoughts commented 1 year ago

In v24 I believe we added a Sequence spaces, this allows an indetermined number of observations

https://gymnasium.farama.org/api/spaces/composite/#sequence

wsdd2 commented 1 year ago

In v24 I believe we added a Sequence spaces, this allows an indetermined number of observations

https://gymnasium.farama.org/api/spaces/composite/#sequence

Thank you. That is a great help, but I am wondering whether gymnasium's env is compatible with openai's imitation, because I want to train the agents using imitation learning.

pseudo-rnd-thoughts commented 1 year ago

For the openai imitation repo, i have no idea but it seems quite old and not updated so i suspect now. I would look at CORL for an up to date, offline RL training library.