Question about the MOPED dataset

[liuyuan-pal]

Hi, thank you for the excellent work. I plan to use the MOPED dataset, but I find the pose is not accurate. I don't know whether I load the data correctly or not.

obj = meshutils.Object3D('data/MOPED/data/toy_plane/reference/integrated_registered_processed.obj')
points = np.asarray(obj.vertices, dtype=np.float32)

input_paths = [x for x in Path('data/MOPED/data/toy_plane/reference').iterdir() if x.is_dir()]
dataset = RealsenseDataset(input_paths,image_scale=1.0,object_scale=1.0,
odometry_type='open3d',center_object=True,use_registration=True)

I assume that the "points" here when projected onto images by the extrinsics given by "dataset" would coincide with the plane. However, they are quite different like the following.

[keunhong]

Hi, can you try using pointcloud.ply instead of that file? The integrated_registered_processed.obj files were centered incorrectly which is causing that slight drift.

I'll update the documentation on the project page to make this clearer, thanks for bringing it up!

[liuyuan-pal]

Thank you for your reply! The scale of pointcloud.ply is not consistent with the scale of integrated_registered_processed.obj. After scaling it manually, the consistency between the model and the image becomes better. But there are still some drifts like the following.

Maybe, the registration between point clouds is not too accurate. Because I find the integrated point cloud of every sequence often coincides with its images better than the global point cloud. Thank you anyway!

[keunhong]

Hi, you have to scale the object by the diameter.

Please see https://github.com/keunhong/latentfusion/blob/master/tools/poserbpf_comparison.py#L199 for the exact parameters.

[liuyuan-pal]

I'm confused about the scales used here. I tried the following codes

data = PlyData.read(str(f'data/MOPED/data/{model_name}/reference/pointcloud.ply'))['vertex']
points = np.stack((data['x'], data['y'], data['z']),1)
diameter = three.points_diameter(torch.from_numpy(points))
object_scale = 1.0 / float(diameter)

input_paths = [x for x in Path(self.root_dir).iterdir() if x.is_dir()]
dataset = RealsenseDataset(input_paths,image_scale=1.0,object_scale=object_scale,
odometry_type='open3d',center_object=True,use_registration=True)

Again, I project the "points" onto images by extrinsics provided by "dataset". And the scale is still incorrect here.

If I manually rescale "point_cloud.ply" so that it has the same diameter as "integrated_registered_processed.obj", then I'll get some reasonable results like the former image.

[keunhong]

Can you try passing in ref_points as in the example I linked above? The centroid has to line up. What the script above does is it uses the raw scale of the input rather than autoscaling. You shouldn't use integrated_registered_processed.obj as that's been scaled and translated arbitrarily and doesn't match up with any extrinsics.

[liuyuan-pal]

pointcloud.ply has already been scaled to the unit sphere. Its diameter is about 1.0. integrated_registered_processed.obj is actually not scaled.

I think I already figured out the transformations between all these coordinates. If I don't scale it (object_scale=1.0), don't offset it (center_object=False) and use registration (use_registration=True), then the output extrinsics are aligned to the model provided by "00/scene/integrated_cropped.ply".

I actually don't need the scaling and offset, I just need the ground truth relative poses among different sequences. Thanks for the reply!

[keunhong]

Hmm, looks like the eval pointcloud I generated didn't make it into the final release for some reason. Let me see if I can fix that.

If you just need to relative poses though then those are available in the registration directory of each capture (the dataset reads this by default).

[keunhong]

Hi, here's a copy of the dataset with the proper pointclouds: https://homes.cs.washington.edu/~kpar/latentfusion/moped.tgz

[liuyuan-pal]

Really appreciate the reply! Thank you! I'll try the new dataset.