Fast Image Matching

[kcoop]

In your comment http://stackoverflow.com/a/11500052/880885 you tantalizingly mentioned you were looking to integrate a GPU version of FREAK into GPUImage, and that you had already done some experimental work with BRISK along these lines. Any chance either of these will make it here? My needs are almost identical to the OP of that question - seems like generically useful functionality for what appears to be a very high performance library.

Ideally what I'm wanting to do is test a real time video stream for matches against a reference image, with moderate scale, shadow and rotation invariance.

[BradLarson]

Yeah, doing feature extraction and matching is my current priority with the framework. I may have a functional FAST-style feature detector soon (which should be a little more performant than my Harris-style corner detectors that I use right now), although I still need to address the slow feature list generation step using something like histogram pyramids.

I've been trying to figure out what to use as a feature descriptor, because I want to avoid SIFT and SURF for patent reasons. BRISK is unencumbered, as I believe is the new FREAK implementation. There's also ORB. My first attempt may just be to replicate Jaco Cronje's GPU-accelerated BFROST approach from this publication:

http://researchspace.csir.co.za/dspace/handle/10204/5387

I'm a little concerned about the number of texture reads required for each descriptor, though, so this might take some tweaking to be fast on the mobile GPUs.

Lastly, I'll need to develop a matcher of some sort and a way to extract homography from that. Mainly, I've seen brute force matchers running on the GPU, so that'll probably be where I start.

[kcoop]

Cool.

FREAK's license is compatible with openCV, so I assume it's unencumbered. Also, in case you hadn't seen it, there's a brief discussion of FREAK's (brute force) matching in the comments here:

http://www.computersdontsee.net/index.php/post/2012/06/25/FREAK-makes-it-into-OpenCV-trunk

[willomitzer]

Dear All,

Yeah, doing feature extraction and matching is my current priority with the framework @BradLarson: Have you had a chance to come back to feature extraction and matching?

[BradLarson]

@willomitzer This is an incredibly difficult problem, and the whole framework is something that I only can work on in my limited free time. You'll know when it's done when I commit the code to the framework. Until then, I make no promises.

[willomitzer]

@BradLarson: Thanks for clarification and good luck (and lot's of free time ;) )

[system123]

Have a look at ColorFAST features. They were implemented for use with GLSL on Android, they also provide a very basic feature descriptor which might be enough for simple correspondance in a tracking like application.

http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=6727003

[BradLarson]

@system123 - Yup, I'm working on integrating that into the framework. I'll implement traditional FAST, but then this. I'm actually a thesis examiner for one of the authors of that paper.

[system123]

Very nice. Really looking forward to it.

[mz2]

@BradLarson did you ever end up implementing image matching on top of GPUImage, or is OpenCV my best bet for now?