Franka Panda Robot Base setting

[sihengz02]

Can I ask why this environment set the robot base position at [-0.615, 0, 0]? If I want to carry on a real robot experiment, how should I calibrate my robot setting?

https://github.com/haosulab/ManiSkill2/blob/fc08823bf96791946591a508f336d724fa7cad26/mani_skill2/envs/pick_and_place/base_env.py#L84

[xuanlinli17]

The base pose was set in a way such that the camera can see the entire robot along with all possible positions of the object.

To calibrate robot setting, I'd recommend to first set up your desired env in real and perform camera calibration. Then, set up a "digital twin" of the real environment in simulation by using the robot base pose and the camera position you got from calibration. Finally, train a policy in sim, and deploy in real.

Another note: you also need to match sim & real in control frequency, robot qvel, etc.

[sihengz02]

Thanks for your timely reply!

Can I further ask what's the difference between arm_pd_ee_delta_pose_align and arm_pd_ee_delta_pose, since this control mode is not introduced in the Docs, but I guess it may be useful for my setting.

https://github.com/haosulab/ManiSkill2/blob/fc08823bf96791946591a508f336d724fa7cad26/mani_skill2/agents/configs/panda/defaults.py#L101

[xuanlinli17]

@Jiayuan-Gu @callmeray

[Jiayuan-Gu]

@Hilbert-Johnson I suggest not using ee_align. The frame defines in which frame the (delta) action space is defined. For example, ee means that the action is relative to the current end-effector frame, base means that the action is relative to the base frame. ee_align means that the delta translation is defined in the ee frame, while the rotation is defined in the base frame.

https://github.com/haosulab/ManiSkill2/blob/fc08823bf96791946591a508f336d724fa7cad26/mani_skill2/agents/controllers/pd_ee_pose.py#L183

[sihengz02]

@Hilbert-Johnson I suggest not using ee_align. The frame defines in which frame the (delta) action space is defined. For example, ee means that the action is relative to the current end-effector frame, base means that the action is relative to the base frame. ee_align means that the delta translation is defined in the ee frame, while the rotation is defined in the base frame.

https://github.com/haosulab/ManiSkill2/blob/fc08823bf96791946591a508f336d724fa7cad26/mani_skill2/agents/controllers/pd_ee_pose.py#L183

Thanks for your clarification! So can I use self.config.frame == "base"? And how to set the self.config.frame to "base"? Since the default self.config.frame seems to be "ee".

[Jiayuan-Gu]

For development, users can just inherit the environment they want to modify and override certain functions. For example, you can override _configure_agent and use your custom AgentConfig (the way is similar to the other way described below). https://github.com/haosulab/ManiSkill2/blob/fc08823bf96791946591a508f336d724fa7cad26/mani_skill2/envs/pick_and_place/base_env.py#L58-L60

Or you can directly modify the file as how we add arm_pd_ee_delta_pose_align https://github.com/haosulab/ManiSkill2/blob/fc08823bf96791946591a508f336d724fa7cad26/mani_skill2/agents/configs/panda/defaults.py#L99-L101 You can also inherit an existing config like https://github.com/haosulab/ManiSkill2/blob/fc08823bf96791946591a508f336d724fa7cad26/mani_skill2/agents/configs/panda/variants.py#L9