eduardohenriquearnold
commented
5 months ago Hi there, I'm glad you find the dataset useful!
I agree the method you mentioned to produce gt correspondences would require a lot of compute in the Map-free dataset.
Other common ways to extract such correspondences more efficiently include back-projecting depth maps from frame A, transforming the points to frame B using relative pose, and projecting them to B's image plane. You would then filter the correspondences by checking their depth approximately matches the depth of the corresponding pixels in B. This approach obviously requires depth maps per frame, which are not available in the Map-free training set.
Perhaps the easiest way to get gt correspondences for the Map-free training set is to use the publicly available SfM models (see colmap models in the download page). There's one model per scene, which includes all images from both scans of each scene. You could get correspondences between frames very easily by checking the common 3D points and their projections for any pair of frames of a given scene. The caveat here is that these models were reconstructed using sift features and, thus, are biased towards those features and are sparser compared to correspondences one gets with LofTR.
I hope this helps, good luck :)
Thanks for your great dataset, it's really awesome and I want to use the dataset to train a RPR network modified from LoFTR.
However, training LoFTR requires correspondences between image pairs, I find that you provided precomputed correspondences for val and test splits. From what I understand, these correspondences are used for produce same pose estimation results as in your paper but not groundtruth.
I have produced some gt correspondences for other datasets by get match pairs with image matching network and filter these match pairs using epipolar distance. But this would be difficult for this dataset, for training set is too large to calculate and filter correspondences from scratch. Is there any method to produce correspondences of training set quickly?