chrischoy / DeepGlobalRegistration

[CVPR 2020 Oral] A differentiable framework for 3D registration
Other
468 stars 85 forks source link

Issure in kitti dataloader #40

Closed gitouni closed 2 years ago

gitouni commented 2 years ago

Thanks for sharing your code. I assume that the input of registration method only includes the input and output point cloud without other additional information. However, in dataloader/kitti_loader.py, the matches = get_matching_indices(pcd0, pcd1, trans, matching_search_voxel_size) uses the ground-truth transformation to get matched indices and this matches are used in WeightedProcrustesTrainer._valid_epoch function in core/trainer.py (input_dict['correspondences']). That means without ground truth pose, the matched indices cannot be acquired and the whole algorithm cannot predict transformation? Maybe I got something wrong, but I'm very confused about this operation in dataloader. Hope for your explanation, thank you. The relative code are shown as follows:

in kitti_loader.py

# Voxelization
    xyz0_th = torch.from_numpy(xyz0)
    xyz1_th = torch.from_numpy(xyz1)

    sel0 = ME.utils.sparse_quantize(xyz0_th / self.voxel_size, return_index=True)
    sel1 = ME.utils.sparse_quantize(xyz1_th / self.voxel_size, return_index=True)

    # Make point clouds using voxelized points
    pcd0 = make_open3d_point_cloud(xyz0[sel0])
    pcd1 = make_open3d_point_cloud(xyz1[sel1])

    # Get matches
    matches = get_matching_indices(pcd0, pcd1, trans, matching_search_voxel_size)  # trans is ground truth transformation

in trainer.py

reg_coords, reg_feats, pred_pairs, is_correct, feat_time, nn_time = self.generate_inlier_input(
          xyz0=input_dict['pcd0'],
          xyz1=input_dict['pcd1'],
          iC0=input_dict['sinput0_C'],
          iC1=input_dict['sinput1_C'],
          iF0=input_dict['sinput0_F'],
          iF1=input_dict['sinput1_F'],
          len_batch=input_dict['len_batch'],
          pos_pairs=input_dict['correspondences']) # relies on matches which need ground truth transformation
gitouni commented 2 years ago

I found that pos_pairs is only used for evluating correct pairs hit.