Question regarding controller `ee_ori_mat` and robot `eef_quat`

[Wuziyi616]

Hi, I have a question regarding the rotations of the controller and robot's eef. When I build the Lift environment, I try to print the rotation of the controller and robot (I'm using robomimic so robosuite is in offline-study branch). I convert them to euler angle, and they seem to have the same x, y angle (difference < 0.1 rad), while differing in z angle by pi/2 rad. On the other hand, their pos are the same, because they're all taken from something like sim.data.site_xpos[self.eef_site_id]. So I wonder why the rotation is different here? I.e. when I want to move the eef to a pose, I have to manually convert that pose from eef frame to the controller frame.

I apologize if this is a stupid problem because I'm new in this area. I'm aware that the difference in rotation is because the controller is taken from site while the eef is taken from body, which are two different elements in Mujoco. My question is more like the motivation under this choice. I go through the document but cannot find anywhere talking about the coordinate system of the controller, eef, hand, etc. (maybe I'm missing something here?)

[Wuziyi616]

BTW another minor question. When I do rotation conversion between rmat and euler, why it's not reversible?

>>> angle = np.array([3.0546, 0.0007, 1.6675])
>>> mat2euler(euler2mat(angle))
array([-3.1325, -0.0865, -1.6679], dtype=float32)

Is it because of the axes parameter here?

[cremebrule]

Hi @Wuziyi616 ,

Thanks for bringing this up -- not a stupid question at all! The rotations are different because the robot currently uses the EEF body (i.e.: gripper body), which is different from the EEF site (i.e.: the actual grasping point).

As far as I know, this is actually a bug. We had the same mismatch with EEF pos before, which we fixed, but apparently we forgot to do that for our EEF orientation as well. @roberto-martinmartin can you confirm this is the case and that we didn't have any other reason why these were explicitly different? Also tagging @amandlek because this likely significantly impacts our robosuite datasets.

re rotation conversions: Multiple euler configurations can result in the same net orientation (which is also the cause for gimbal lock), so I don't think the conversion is an invertible process (since it's not a 1-to-1 mapping). @roberto-martinmartin likely has better insight.

[Wuziyi616]

@cremebrule Thank you so much for the reply, that helps a lot! Yes, I understand there is ambiguity in Euler angle, which solves my second question.

Regarding the rotation, assume I'm using the IK controller, the document says "the rotation axes are taken relative to the end-effector origin". Then if I want to move the EEF to a pose (R, T) (T is the translation i.e. xyz coordinate of that position, and R is the rotation matrix of this pose), how should I construct the action input to env.step()? For its first three items, I think it's the relative movement, T - eef_pos. For the rotation elements, is it mat2axisangle(quat2mat(eef_quat).T @ R)?

EDIT: BTW, if I'm using OSC controller and the rotation input is "relative to the global world coordinate frame", then is the action input mat2axisangle(R @ quat2mat(eef_quat).T)?

[Wuziyi616]

@cremebrule Hi, just FYI, I can also confirm the rotation difference only happens on Panda, not on other robots e.g. Sawyer