search

jr-robotics / robo-gym

An open source toolkit for Distributed Deep Reinforcement Learning on real and simulated robots.
https://sites.google.com/view/robo-gym
MIT License
390 stars 74 forks source link

How to reset to a specific state #31

Closed Jiahui-3205 closed 3 years ago

Jiahui-3205 commented 3 years ago

Hi, we are using robo-gym on imitation learning. We are using the simulated environment 'EndEffectorPositioningURSim-v0', ur3. It's a very powerful tool.

In our case, we want to reset the environment to a specific initial state. We reset the environment side, but on the server-side we cannot reset to the same state. Is there a method that can reset to an initial state on both robo-server side and the environment side?

Thanks

matteolucchi commented 3 years ago

Hi @Jiahui-3205 ! I am happy to hear that you are using robo-gym and thanks for your kind words!

If I understand correctly your request, you want to reset the robot to specific joint positions and the target to a specific position. That can be done by passing the joint positions and the End Effector target pose to the reset method.

As you can see in the method definition https://github.com/jr-robotics/robo-gym/blob/9be636ed577fdd6376c65152ba566fe6515709cf/robo_gym/envs/ur/ur_ee_positioning.py#L173-L180

Let me know if this solves your issue :)

Cheers,

Matteo

Jiahui-3205 commented 3 years ago

Thank you for the reply. I use the variable "ee_target_pose" from the initial reset function as input of the new reset function and manage to reset. But the new reset state has little difference from the original state. Then I found the variable "rs_state" has little difference from the origin "rs_state", and cannot get exactly the same. I think this may cause the output of the reset different from the initial reset state. But currently, the output of the reset function is acceptable, and thanks for the help.

matteolucchi commented 3 years ago

Hi,

Then I found the variable "rs_state" has little difference from the origin "rs_state", and cannot get exactly the same.

Yes this is because of some 'noise' in the Gazebo simulation, the state of simulation is set as per described in the rs_state which is sent to the robot server, then the robot server gets the state of the simulation via different ROS topics published directly by Gazebo or simulated sensors, and due to some noise there is always a little difference between the state which is set and the state which is read.

I think this may cause the output of the reset different from the initial reset state.

Exactly, you are right with that.

But currently, the output of the reset function is acceptable, and thanks for the help.

That's great to hear, glad I could help!

Cheers, Matteo