search

GrumpyZhou / image-matching-toolbox

This is a toolbox repository to help evaluate various methods that perform image matching from a pair of images.
MIT License
543 stars 80 forks source link

nn_matching: cannot perform reduction function max on tensor with no elements #17

Closed v-pnk closed 3 years ago

v-pnk commented 3 years ago

Hello @GrumpyZhou ,

Image pair matching sometimes fails with the following error:

File "my_code.py", line 111, in <lambda>
    matcher = lambda im1, im2: model.match_pairs(im1, im2)
File ".../image-matching-toolbox/immatch/modules/caps.py", line 79, in match_pairs
    matches, kpts1, kpts2, scores = self.match_inputs_(im1, gray1, im2, gray2)
File ".../image-matching-toolbox/immatch/modules/caps.py", line 69, in match_inputs_
    match_ids, scores = self.mutual_nn_match(desc1, desc2, threshold=self.match_threshold)
File ".../image-matching-toolbox/immatch/modules/base.py", line 63, in mutual_nn_match
    return mutual_nn_matching(desc1, desc2, threshold)
File ".../image-matching-toolbox/immatch/modules/nn_matching.py", line 28, in mutual_nn_matching
    matches, scores = mutual_nn_matching_torch(desc1, desc2, threshold=threshold)
File .../image-matching-toolbox/immatch/modules/nn_matching.py", line 12, in mutual_nn_matching_torch
    nn12 = similarity.max(dim=1)[1]
RuntimeError: cannot perform reduction function max on tensor with no elements because the operation does not have an identity

My guess is that there is an image, for which no keypoints are detected (yes, there is one such image), desc1 or desc2 array is empty, similarity matrix is empty and therefore the max operation fails. Implementing a test somewhere, checking if desc1 or desc2 are not empty (and dealing with the empty ones) would be great. I've seen the issue only with CAPS, but it could probably appear also for other methods.

Thank you very much!

v-pnk commented 3 years ago

The same issue appeared with R2D2. Tried to implement quick dirty fix in caps.py and r2d2.py by detecting empty kpts1 or kpts2 and then returning empty matches and scores, which seems to work.

GrumpyZhou commented 3 years ago

Hi @v-pnk , thanks for reporting the error can you try it again?

v-pnk commented 3 years ago

Hi @GrumpyZhou , It seems to work OK now. Thank you very much for your quick response!