How to reset the environment?

[kuroko1789]

Hi,

I would like to use PyRep as a reinforcement learning environment, so I try to follow the example found here. However, I don't know how to reset the environment properly. I want to reset youbot to the initial state, so inside my reset function, I call

self.agent.set_2d_pose(pos)
self.agent.set_base_angular_velocities([0,0,0])
return self._get_state()

It seems that the return sensor data is from the last step before reset, and the robot joint velocity is far from zero. How can I reset the environment, do I need to call vrep.simStopSimulation and vrep.simStopSimulation or pr.stop() and pr.start()? I tried to call pr.stop() and pr.start() inside my reset() function, but the simulation acts weird. How can I restart simulation using pyrep? Is there a better way to reset except stopping the simulation and restart again?

Thanks!

[stepjam]

Hi @kuroko1789

After you perform the reset, you should step the sim and then your sensor data should be correct. It may be worth setting the target velocities to 0 on a reset in your case too.

Yes, pr.stop() and pr.start() is indeed another way of doing this. If you do this approach, it is worth calling pr.step() inbetween those calls so v-rep can do some internal stuff.

Please let me know if this helps.

[kuroko1789]

Hi Stephen,

I set the target velocities to 0, and use get_joint_velocities to get the joint velocities. Based on the return value, the velocities appear not to be reset. My code is as follows:

def reset(self):
    self.agent.set_2d_pose(self.starting_pose)
    self.agent.set_joint_target_velocities([0,0,0,0])   
    self.pr.step()
    return self._get_state()

[stepjam]

Hi @kuroko1789 , Sorry for delay in getting back to you. Did you manage to resolve this? If not, could you check what the control mode is? It should be torque/force mode when using velocity control.

Try this at the start of the simulation perhaps?

agent.set_joint_mode(JointMode.FORCE)
agent.set_motor_locked_at_zero_velocity(True)

[kuroko1789]

Hi,

It works after adding set_joint_mode. Thanks.

[stepjam]

Glad that worked :)