search

NVlabs / DREAM

DREAM: Deep Robot-to-Camera Extrinsics for Articulated Manipulators (ICRA 2020)
Other
149 stars 33 forks source link

Another questions regarding keypoints' location in the coordinates of the corresponding ros_frame #25

Closed Oliverbansk closed 1 year ago

Oliverbansk commented 1 year ago

Hello! Thank you for your explanation before! I actually have another question about keypoint's 3D location in the coordinates of the corresponding ros_frame.

image

In ./manip_configs/kuka.yaml , there is declaration of the keypoints as well as each keypoint's corresponding link and ros_frame. I'm now trying to figure out the 3D location in the coordinates of the corresponding ros_frame of each keypoint. Is it safe to deduct that the frame's origin is placed exactly in the keypoint's location, so the coordinates I've been trying to find is actually (0,0,0)?

tabula-rosa commented 1 year ago

Hi Shikun! Thanks for your continued interest in our work!

For our work, we assumed that the robot keypoints were coincident with the robot joints. So, it should be the case that each keypoint's 3D location should be at the origin of the specified ros_frame. Hope that helps!

Oliverbansk commented 1 year ago

Hi! Thank you for your explanation! I actually would like to know more about your definition of ros_frames. What convention did you follow when assigning those frames? I'm currently using other fk calculation tools to compute fk of each links/joints. They seem to place their link_frames at the joints. Is your ros_frame also placed at the joint location too? or at the geometry center of the links?

tabula-rosa commented 1 year ago

The ros_frames field is only used by the DREAM ROS node. It is used in the following ways:

In short, the ros_frame fields are the TF lookup names for each keypoint's coordinate frame in the ROS TF tree. It is assumed that the URDF used for defining the keypoints during training is the same URDF that is used by ROS.

The convention used was we assume there is a keypoint at the coordinate frame origin of each link, as based on the robot URDF. (Strictly speaking, the keypoints could be anywhere on the robot, it's just that for our work, we chose to place them at the origin of each link.) According to the original Franka Emika Panda URDF, some of the links have coordinate frames defined at unintuitive locations, leading to the keypoint for link1 and link2 being the same. This is also true for link5 and link6. This is why there are only 7 keypoints for the Franka DREAM networks that were released, not 9. Each of the 7 keypoints are at a joint --- it's just that there is no keypoint at the joint between link0/link1 and link4/link5.

For your FK tools, are they based on the original Panda URDF and have the coordinate frames for link1/link2 and link5/link6 sharing the same position in 3D space? Or do they correct for this under the hood of the software? I used a FK tool when generating the real datasets without issue, so with the original Panda URDF it should be consistent with the released DREAM networks.

Oliverbansk commented 1 year ago

Thank you for your thorough explanation!!! The fk tools I have tried actually just conduct mathematical computing of the FK, whether the coordinate frames for link1/link2 and link5/link6 share the same position in 3D space is to my opinion depending on the URDF. I'm wondering if you can you share your source (github repo or ros_wiki) of your original Franka Emika Panda URDF (maybe baxter and kuka's as well if it's convenient to get..), I'm afraid that I have chosen the wrong version so that it leads to the inconsistent result... Thank you for your detailed elaboration again!!

tabula-rosa commented 1 year ago

Unfortunately, I no longer have access to the URDF files for the robots, as I no longer have access to the files related to my internship when I worked on this project. So, I'm afraid that I am unable to share them.

However, the URDF definition of the Franka Emika Panda robot is available at their public repo: https://github.com/frankaemika/franka_ros/tree/develop/franka_description

The definition is provided in URDF xacro format, so you may need to install the ROS package and run the examples on this website to verify it.

It's possible that in the URDF, the Panda joints in question are defined with some position offset with respect to their parent link. So, if you find that offset, you can use the TF lookup for link1 and apply this offset to recover where the position of joint1 is. Same for link5/joint5. However, I would expect the joint positions for the rest joints to be the same as the coordinate frame origin for link0, link2, link3, link4, link6, link7, and the hand. Hope that helps!

Oliverbansk commented 1 year ago

Thank you very much!!!!