search

ir413 / mvp

Masked Visual Pre-training for Robotics
214 stars 25 forks source link

Reset management for multiple envs #7

Closed MathisClautrier closed 1 year ago

MathisClautrier commented 2 years ago

Hi, thanks for this project and making available your code,

I am playing with PixMc and your model for few days, but I still have a pending question,

In task/configs an episode length is specified whereas in mvp/ppo.py the environments are never reinitialized (apply_reset set to false). I am deducing that this is automatic. Similarly, I assume that when a given environment reaches a goal it is reinitialized automatically. Am I right to think so? And if not, how can I successfully reset the environment when the goal states are reached?

ir413 commented 1 year ago

Hi @MathisClautrier, sorry for the late response. In case this is still useful: we use reset_buf tensors to keep track of resets. The environments are reset when the max episode length is exceeded or one of the other termination criteria is met (e.g., arm collision or object below the table). We do not reset the environments when the goal is reached. Hope this helps!