search

google-research-datasets / Objectron

Objectron is a dataset of short, object-centric video clips. In addition, the videos also contain AR session metadata including camera poses, sparse point-clouds and planes. In each video, the camera moves around and above the object and captures it from different views. Each object is annotated with a 3D bounding box. The 3D bounding box describes the object’s position, orientation, and dimensions. The dataset contains about 15K annotated video clips and 4M annotated images in the following categories: bikes, books, bottles, cameras, cereal boxes, chairs, cups, laptops, and shoes
Other
2.24k stars 263 forks source link

Loading poses into COLMAP #66

Open weders opened 2 years ago

weders commented 2 years ago

I need to load the poses obtained from this dataset into COLMAP. While I am aware of this thread, following it does not yield satisfactory results.

Given the convention, one would need to invert the y-axis and z-axis of the pose such that it aligns with the CV camera convention (x right, y down, z in camera viewing direction). However, this does not give correct results. Is any preprocessing happening between raw ARKit poses and the ones stored in the geometry.pbdata file?

Thanks!

ahmadyan commented 2 years ago

The geometry.pbdata contains the original ARKit camera poses.

@lzhang57 did this the other way around (bringing COLMAP camera poses to our coordinates without any problem. The result are written in sfm_arframe.pbdata file next to geometry.pbdata. The pose/point cloud in sfm_arframe.pbdata is computed using COLMAP then converted to our coordinate system.

weders commented 2 years ago

Thanks a lot for the answer! Is this code for the COLMAP conversion available?

lzhang57 commented 2 years ago

Hi weder,

When converting camera coordinates from COLMAP to our convention, we left multiplied the view_matrix with the below adjust_matrix, which swaps x and y and inverts z:

adjust_matrix = np.array(
    [[0.,   1.,   0.],
     [1.,   0.,   0.],
     [0.,   0.,  -1.] ])

You can just do it reversely.