search

wenbowen123 / icra20-hand-object-pose

[ICRA 2020] Robust, Occlusion-aware Pose Estimation for Objects Grasped by Adaptive Hands
Other
52 stars 6 forks source link

Using a new hand with known configuration #3

Closed hongtaowu67 closed 3 years ago

hongtaowu67 commented 3 years ago

Hi Bowen,

It might be not very related to this paper. But I have a Robotiq 2f-85 hand. I have the 3d CAD model of each part of the hand. Thus, I can actually generate a point cloud for each part individually. Also, I can read the configuration of the finger from the encode. In this figure, it seems that you are able to merge point clouds from different part into a complete one for tracking. I am wondering how you did that?

Thanks!

wenbowen123 commented 3 years ago

Hi @hongtaowu67

In the hand we are using, each finger is treated as an articulated object (there is an additional joint in the middle of the finger). The figure you pointed is generated by saving the point cloud of each components together for visualization. In practice, we are still treating them as articulated object rather than merging, i.e. apply transformations to each links while satisfying articulated DOF.

hongtaowu67 commented 3 years ago

Thanks for the reply. I see so you are treating each part individually and apply transformations to put them together before registration. But what about the points which is occluded by the joint (e.g, the joint between the base and the finger will occlude some points in the finger link)? Did you remove them manually or it doesn't matter in terms of registration.

wenbowen123 commented 3 years ago

For estimating the hand's state, each finger's mesh is registered with the observed point cloud individually. So the intersection example you give won't affect the finger's mesh or the observed point cloud, since the internal intersection is not observable anyway (the joint as the intersection part between base and finger). Also the joint's intersection region is very small compared with the size of the finger or the handbase. Therefore, this kind of occlusion probably won't affect the registration process and we didn't remove them manually.

If you are using Robotiq 2f-85 hand, then it's even better because you now know the hand's state from FK, the hand state estimation step can be skipped.

hongtaowu67 commented 3 years ago

I see. Thank you! I will try it. I am closing this issue.