How to get initial object poses in camera frame

[ChrisSun99]

Hi Bowen, from my understanding, ob_in_cam gives the relative pose wrt the initial object pose in camera frame (aka what 0000.txt provides in /annotated_poses). In our setting, we use AprilTags to get the object pose. However, I'm not seeing any markers in your demo. How did you get the object pose? Thank you.

[wenbowen123]

Hi, the concept is similar to BundleTrack. The tracking tracks the delta motions of the object relative to the first frame (how much does the object translate/rotate along the video). We plotted the axis aligned bounding box only for visualization purpose. For a novel unseen object, the coordinate system setup can be arbitrary (you can make XYZ axis pointing any direction as long as they are perpendicular).

In your case, since you have AprilTags, seems like you default to that coordinate system. You can do this operation to transform all the pose output to reflect this defined coordinate system

final_poses = ob_in_cams@np.linalg.inv(ob_in_cams[0])@april_tag_pose