Matching against multiple images

[catree]

Hi,

I am wondering if this method is suitable for matching one image against multiple images. What I want to achieve is object pose estimation. Typically I do:

"Training" step:

• take multiple images of the desired object
• detect keypoints and extract descriptors only on the object
• store in a file the 2D image locations, the corresponding descriptors and the 3D object coordinates

Detection step:

• detect keypoints and extract descriptors in the current image
• match current keypoints with those detected on the training images (current keypoints matched with training keypoints concatenated)
• use the ratio test to try to discard most of the false matches
• compute the object pose with the 2D-3D correspondences with a PnP algorithm and a RANSAC scheme

With LOGOS, I was thinking about: "Training" step:

• similar steps 1), 2) and 3)
• in addition, take some images relevant to the expected background with and without the object
• detect keypoints and descriptors and perform Bag of Words
• assign a BoW label to the training keypoints/descriptors

Detection step:

• similar steps 1) and 2)
• assign a BoW label to the current keypoints
• perform matching with LOGOS
• PnP pose estimation with RANSAC

Do you think the LOGOS method is appropriate to match N keypoints detected in the current image with M keypoints (concatenation of keypoints detected on multiple images and thus with different object views)? Also, is my approach valid?

Thanks.

[short-circuitt]

Hi,

With anything there is of course a trade-off. If you store a Bag of Words instead of the full descriptors then it will require less memory, but there will almost certainly be a drop in performance. It really depends what it more important for the particular application.

Similarly, LOGOS makes some simplifying assumptions about the positions of the descriptors. This allows it to use smaller descriptors in combination with a coarse Bag of Words (which was appropriate to the related application). I'm not sure if these assumptions will work in an application where the descriptors come from multiple images taken from different poses, it will really be specific to the particular application.

[catree]

Thanks for the answer.

I think I got it. This method is intended for matching against a large number of images where database descriptors are not available and reduced to BoW code to reduce memory footprint and computation. For applications where query and trained descriptors are available, classical matching strategies like ratio test should be adopted.

This is the matching result with LOGOS, OpenCV SURF and BoW:

There are some performance degradations that could come from a different implementation between OpenCV and Matlab SURF implementations and VLAD is not used here.

I would like to add this method in OpenCV contrib module if you have no objection.

[short-circuitt]

Yes of course!

[FredHuang16]

Thanks for the answer.

I think I got it. This method is intended for matching against a large number of images where database descriptors are not available and reduced to BoW code to reduce memory footprint and computation. For applications where query and trained descriptors are available, classical matching strategies like ratio test should be adopted.

This is the matching result with LOGOS, OpenCV SURF and BoW:

There are some performance degradations that could come from a different implementation between OpenCV and Matlab SURF implementations and VLAD is not used here.

I would like to add this method in OpenCV contrib module if you have no objection.

Could you please share your implementation code ? Thanks very much.

[catree]

@FredHuang16 An implementation code is available in OpenCV contrib, pull request.