XiaoshuiHuang / IMFNet

[IEEE RAL 2022] IMFNet: Interpretable Multimodal Fusion for Point Cloud Registration
MIT License
46 stars 9 forks source link

Correspondences present in the result file IMFNet_3DMatch_result do not seem to be accurate #2

Open sidch0 opened 2 years ago

sidch0 commented 2 years ago

In the result folder IMFNet_3DMatch_result, taken from (https://drive.google.com/drive/folders/1Pb9bkQefwgBfxcrrfUokiY7_EYv10dfD?usp=sharing), the corresponding keypoints given for pairs of scans in the IMFNet subfolder don't seem to be accurate.

The corresponding keypoints are numpy array files which have two sets of indices inds_i and inds_j in each file. I assume that they are the keypoint ID's of the matched correspondences between the two scans 'i' and 'j'. For example, I presume that in the folder, "IMFNet_3DMatch_result/IMFNet", the file "7-scenes-redkitchen_seq-01_0_1_keypoints.npz" is the set of correspondences 'inds_i' and 'inds_j' between the scans "cloud_bin_0.ply" and "cloud_bin_1.ply" in the kitchen dataset in the 7-scenes-redkitchen dataset taken from http://vision.princeton.edu/projects/2016/3DMatch/downloads/scene-fragments/7-scenes-redkitchen.zip .

I assumed that the keypoint_ids given by inds_i and inds_j are the indices of the points in the point clouds "cloud_bin_0.ply" and "cloud_bin_1.ply" respectively. But there seems to be a mismatch in the correspondences on visual inspection and they don't seem to give good registration results also. Are there are any conventions and details that I am missing?

Also, the subfolders IMFNet and IMFNet_keypoints both seem to contain exactly the same .npz files. For example, the files "IMFNet/7-scenes-redkitchen_seq-01_0_1_keypoints.npz" and "IMFNet_keypoints/7-scenes-redkitchen_seq-01_0_1_keypoints.npz" have the same keypoint_id pairs inds_i and inds_j.

QWTforGithub commented 2 years ago

Also, the subfolders IMFNet and IMFNet_keypoints both seem to contain exactly the same .npz files. For example, the files "IMFNet/7-scenes-redkitchen_seq-01_0_1_keypoints.npz" and "IMFNet_keypoints/7-scenes-redkitchen_seq-01_0_1_keypoints.npz" have the same keypoint_id pairs inds_i and inds_j.

Thank you for your interest in our work. The result after running evaluation_3dmatch.py should contain 5 files: IMFNet, IMFNet_keypoints, IMFNet-metrics-0.10.csv, IMFNet-metrics-0.10.pkl, recall-0.10.pdf. The IMFNet folder stores the files of scene result files. IMFNet_keypoints is used to save key points, that is, the index of the points. If you want to use keypoints to run the results, just put the IMFNet_keypoints folder in the output folder, then keypoints=True. The result folder of the Google drive web, I have re-uploaded. On our machine can produce a very good FMR and RR. Did you have a poor result on your machine?

QWTforGithub commented 2 years ago

In the result folder IMFNet_3DMatch_result, taken from (https://drive.google.com/drive/folders/1Pb9bkQefwgBfxcrrfUokiY7_EYv10dfD?usp=sharing), the corresponding keypoints given for pairs of scans in the IMFNet subfolder don't seem to be accurate.

The corresponding keypoints are numpy array files which have two sets of indices inds_i and inds_j in each file. I assume that they are the keypoint ID's of the matched correspondences between the two scans 'i' and 'j'. For example, I presume that in the folder, "IMFNet_3DMatch_result/IMFNet", the file "7-scenes-redkitchen_seq-01_0_1_keypoints.npz" is the set of correspondences 'inds_i' and 'inds_j' between the scans "cloud_bin_0.ply" and "cloud_bin_1.ply" in the kitchen dataset in the 7-scenes-redkitchen dataset taken from http://vision.princeton.edu/projects/2016/3DMatch/downloads/scene-fragments/7-scenes-redkitchen.zip .

I assumed that the keypoint_ids given by inds_i and inds_j are the indices of the points in the point clouds "cloud_bin_0.ply" and "cloud_bin_1.ply" respectively. But there seems to be a mismatch in the correspondences on visual inspection and they don't seem to give good registration results also. Are there are any conventions and details that I am missing?

Also, the subfolders IMFNet and IMFNet_keypoints both seem to contain exactly the same .npz files. For example, the files "IMFNet/7-scenes-redkitchen_seq-01_0_1_keypoints.npz" and "IMFNet_keypoints/7-scenes-redkitchen_seq-01_0_1_keypoints.npz" have the same keypoint_id pairs inds_i and inds_j.

If you still produce a bad RR and FMR result, please try it without key points.