NVlabs / FoundationPose

[CVPR 2024 Highlight] FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objects
https://nvlabs.github.io/FoundationPose/
Other
955 stars 99 forks source link

About the coordinate frame of target object #72

Closed ZisongXu closed 3 weeks ago

ZisongXu commented 4 weeks ago

Hi Dear, I have a question about the coordinate frame of the result of the target object.

This is the model I feed into the FoudationPose system, and we can see the coordinate frame clearly. Screenshot from 2024-04-19 15-19-38

But what I get in the result is: Screenshot from 2024-04-14 19-45-54

We can see the difference between the coordinate frame of .obj model file and the coordinate frame of result.

And I also check the example you provided, this is the model:

Screenshot from 2024-04-19 15-27-28

This is the result:

Screenshot from 2024-04-19 15-26-13

Also different, is that right or should I set the coordinate frame some where before running FoudationPose system?

Thanks a lot! Best Regards Zisong XU

ZisongXu commented 4 weeks ago

By the way, about the results, we can get the object pose in camera. Can you tell me the coordinate frame of the camera. Because I need to convert the object pose in camera to object pose in world frame so I can use this system.

If I'm not mistaken, the z-axis of the camera is looking towards the photo, the y-axis is down, and the x-axis is to the right

wenbowen123 commented 3 weeks ago

You can disable this line https://github.com/NVlabs/FoundationPose/blob/8395bd84adb0bfc3209cb9409e475cd1b51fc17b/run_demo.py#L69 which may change the coordinate.

However, the pose being saved is still correct https://github.com/NVlabs/FoundationPose/blob/8395bd84adb0bfc3209cb9409e475cd1b51fc17b/run_demo.py#L66

wenbowen123 commented 3 weeks ago

it's using OpenCV camera coordinate system: https://stackoverflow.com/questions/17987465/how-is-the-camera-coordinate-system-in-opencv-oriented

ZisongXu commented 3 weeks ago

Thanks a lot!