search

NVlabs / FoundationPose

[CVPR 2024 Highlight] FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objects
https://nvlabs.github.io/FoundationPose/
Other
1.19k stars 143 forks source link

Real-time estimation issue where the object diverges when it goes out of bounds #176

Open SeungHunJeon opened 2 weeks ago

SeungHunJeon commented 2 weeks ago

Hi, thank you for sharing your great works !

I attempted to perform real-time pose estimation using RGBD sensor data from the D435.

However, I encountered an issue where the object diverges and does not re-converge once it goes out of the scene.

How can this problem be resolved?

https://github.com/NVlabs/FoundationPose/assets/54147180/c792f60c-cd1a-463b-877b-6f43bd902f88

Kaivalya192 commented 2 weeks ago

Hi, thank you for sharing your great works !

I attempted to perform real-time pose estimation using RGBD sensor data from the D435.

However, I encountered an issue where the object diverges and does not re-converge once it goes out of the scene.

How can this problem be resolved?

IMG_9041.online-video-cutter.com.mp4

hey there, i think you are using the existing logic of this repo that registers pose for the first frame and then for next all frame it estimates pose.

when it is out of camera send the pose to score network and see the score. Then set some threshold accordingly. if threshould is crossed then reregister the pose and pass the mask for that.

as mentoined in #136

SeungHunJeon commented 2 weeks ago

Ok then, I need to stop the tracking and re-estimate the pose accordingly. I think to resolve that issue in real-time, masking should be processed automatically, not manually.

assia855 commented 2 weeks ago

how can I do that please @Kaivalya192 ?