search

NVlabs / FoundationPose

[CVPR 2024 Highlight] FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objects
https://nvlabs.github.io/FoundationPose/
Other
1.35k stars 174 forks source link

The lost of tracking #131

Closed Ginger-X closed 1 month ago

Ginger-X commented 4 months ago

The first half of the target object is tracked well, but after masking or moving out, the position estimation is no longer carried out correctly, any ideas on how to solve this, thanks!

wenbowen123 commented 4 months ago

do you have a video showing this?

Ginger-X commented 3 months ago

yes, I have. I know that the position estimation is inevitably poor when there is heavy occlusion, but do you have any thoughts on tracking the position again when the target object reappears, e.g. with the help of the detection model. Also, I would like to ask how the AR effect is achieved, can you provide demo code, thanks a lot!

https://github.com/NVlabs/FoundationPose/assets/65015349/e682e24c-f266-4efa-ad84-a8a006985c99

wenbowen123 commented 3 months ago

https://github.com/NVlabs/FoundationPose/issues/136 maybe related

Ginger-X commented 3 months ago

Thank you for your reply, it will be very helpful. Also, I would like to ask how the AR effect is achieved as shown in demo, can you provide example code or describe the tools used to achieve this? thanks a lot!

wenbowen123 commented 3 months ago

@Ginger-X unfortunately this part involves some other internal modules beyond FoundationPose, which is not sharable for this point. But once you have the 6D pose, the other parts are only on the engineering side.