search

benelot / pybullet-gym

Open-source implementations of OpenAI Gym MuJoCo environments for use with the OpenAI Gym Reinforcement Learning Research Platform.
https://pybullet.org/
Other
814 stars 124 forks source link

Fixed Ant never sending the done signal #67

Open sash-a opened 3 years ago

sash-a commented 3 years ago

Noticed this when I was trying to find a fix for this issue. The bug seems to be with these two lines:

https://github.com/benelot/pybullet-gym/blob/bc68201c8101c4e30dde95f425647a0709ee2f29/pybulletgym/envs/mujoco/robots/locomotors/ant.py#L29

and

https://github.com/benelot/pybullet-gym/blob/bc68201c8101c4e30dde95f425647a0709ee2f29/pybulletgym/envs/mujoco/envs/locomotion/walker_base_env.py#L59-L60

I'm not sure if it should be changed in Ant or in WalkerBaseMuJoCoEnv not sure if other envs could benefit from this too? I made the change in Ant so as not to affect other envs.

Steps to reproduce the bug, simply run:

env = gym.make('AntMuJoCoEnv-v0')
env.render()
env.reset()
done = False
step = 0
while not done:
    _, _, done, _ = env.step([0] * 8)
    step += 1

print(step)

It should print 1000 indicating that the Ant run for the max number of steps, but its torso should have been touching the ground the whole time, which is undesired behavior.

benelot commented 3 years ago

Sounds like a reasonable fix for now. I think I will have to go through the code again at some point. I did not really find time and motivation as I was stuck with the missing observations anyway, making the project hard to use compared to mujoco. Maybe there will be a resolution to all this soon.