Poses in MegaPose-GSO can't be matched to images

[Liu-Xinhang]

I find some mismatches in the MegaPose-GSO. I read the pose of each object from the gt.json file. The translation is about three-digit, so i guess the unit is mm. Then I load the corresponding model and I found the diameter is about two-digit decimal，so I guess the unit is m. Then I read the intrinsic-matrix from the camera.json file. When I project the model to the image using these data after adjusting the unit, I found the size seems correct but the location has a mismatch. So what's wrong with my manipulations.

[ylabbe]

The GSO models need to be scaled by 0.1, see https://github.com/thodan/bop_toolkit/blob/master/docs/bop_challenge_2023_training_datasets.md

[Liu-Xinhang]

I found the GSO models are in two-digit decimal. How could it match the pose which has three digits in translation, by scaling by 0.1? Is there something I missing?

[ylabbe]

You are actually correct the scaling that needs to be applied to the original meshes is not just 0.1 ! (See #100 and https://github.com/megapose6d/megapose6d/blob/master/src/megapose/scripts/make_gso_meshes.py#L93). The meshes were centered and normalized to fit within unit sphere before applying the 0.1 scaling. The pseudo code to generate the correct meshes is:

xmin, xmax = float(vertices[:, 0].min()), float(vertices[:, 0].max())
ymin, ymax = float(vertices[:, 1].min()), float(vertices[:, 1].max())
zmin, zmax = float(vertices[:, 2].min()), float(vertices[:, 2].max())
scale = max(max(xmax - xmin, ymax - ymin), zmax - zmin) / 2.0

vertices[:, 0] -= (xmax + xmin) / 2.0
vertices[:, 1] -= (ymax + ymin) / 2.0
vertices[:, 2] -= (zmax + zmin) / 2.0
vertices[:, :3] /= scale 
vertices[:, :3] *= 0.1

Please let me know if that works now, really sorry about that. We will update the dataset description accordingly.

[Liu-Xinhang]

Thanks and sorry about for the late reply. This works fine with the change in ground truth translation from mm to m.