How do I get the true pose of the custom object in camera coordinates

[robot-xbrain]

I can now find the bounding box of my custom object in the image, but how do I get the object's pose in camera coordinates.I'm confused

[liuyuan-pal]

Hi, The pose is [R;t] which means x_camera = R @ x_object + t. The object points in object coordinate system (x_object) are a rotated version of your object_point_cloud.ply as described in https://github.com/liuyuan-pal/Gen6D/issues/14#issuecomment-1224089952

[robot-xbrain]

Sorry to bother you.I'm not quite familiar with this piece of knowledge.In my view, the formula pts = np.matmul(pts,RT[:,:3].transpose())+RT[:,3:].transpose() from the function project_points of file base_utils.py can get the object's pose in camera coordinates.I hope to get your help

[robot-xbrain]

I mean I can get the vertex coordinates of the object in camera coordinates through the formula pts = np.matmul(pts,RT[:,:3].transpose())+RT[:,3:].transpose().But the depth results don't seem right,I'm still confused

[liuyuan-pal]

The z value of all projected points are depth values for these points. Maybe, you want the depth value in meters. However, these depth values are not in the standard length metric meter because the object is reconstructed from reference images so the reconstructed points have ambiguity in scale.

[robot-xbrain]

The z value of all projected points are depth values for these points. Maybe, you want the depth value in meters. However, these depth values are not in the standard length metric meter because the object is reconstructed from reference images so the reconstructed points have ambiguity in scale.

yes,I want the depth value in meters. Can I easily get this value from your code, and if so, how ?Thank you for your reply

[liuyuan-pal]

There is scale ambiguity in multiview 3D reconstruction, so getting depth in meters is impossible. You have to scale depth values manually.

[apirrone]

Let's say I know the real size (along one axis) of the object in meters. I also know the length of the bounding box along the same axis in the arbitraty units of the networks. How can I use this information to get the translation part of the pose matrix in meters ?

I tried multiplying the translation by the ratio between the actual length (in meters) and the bounding box size, but I the values I get are very far from reality.

Thanks !

[apirrone]

My mistake, the values actually look quite right ... I posted too quick sorry :)