multi-face support

[kylemcdonald]

this is definitely possible, and might go along with better re-initialization algorithms. doing a manual haar face detection pass and using the results to initialize multiple face trackers, rather than relying on the internal haar detector, would help a lot in this direction.

[cpietsch]

are there any news on the multi-face support ?

[kylemcdonald]

no updates. it's probably not going to happen until i need it for a project, or until someone hires me to implement it so they can use it for their project.

[freber]

I'm interested in trying to implement a multiple face tracker solution. Were would you recommend me to start?

[kylemcdonald]

The next step would be to replace Jason's face detector with the opencv object detector. Then you would need to create one instance of a face tracker for each detected face. It will be a bit slow because object detection is very slow (whenthe tracker stalls while looking for a face, it would be like that all the time). You could run the detector in a separate thread and then feed all the most recent frames to the recognizer to catch it up. But anyway, the first step is replacing Jason's detector with the opencv detector, and I'm not sure it will be so fast in the end.

[cpietsch]

Do you mean with "opencv object detector" the cvMatchTemplate ? I have done that in a thread and a storage for each face which was detected in the first hand via CascadeClassifier. But it is still a bit buggy and unfinished.

[freber]

Post it Christopher! :) — Sent from Mailbox for iPhone

On Mon, May 13, 2013 at 12:22 AM, Christopher Pietsch notifications@github.com wrote:

Do you mean with "opencv object detector" the cvMatchTemplate ?

I have done that in a thread and a storage for each face which was detected in the first hand via CascadeClassifier. But it is still a bit buggy and unfinished.

Reply to this email directly or view it on GitHub: https://github.com/kylemcdonald/ofxFaceTracker/issues/24#issuecomment-17786499

[kylemcdonald]

@cpletsch i mean the CascadeClassifier, not the template matcher. the template matcher is incredibly slow, and not robust for face tracking. if you're using opencv straight up, CascadeClassifier is good (and i would recommend submitting to the http://github.com/kylemcdonald/FaceTracker repo instead). if you're using ofxCv you could use https://github.com/kylemcdonald/ofxCv/blob/master/libs/ofxCv/include/ofxCv/ObjectFinder.h

[cpietsch]

@kylemcdonald before my attempt rottens away.. i put it on https://github.com/cpietsch/ofxTemplateFace . its a bit like your object finder class (handling rescaling) but uses cvTemplateMatch to re-track the face. not that un-performant and quite accurate but not really robust :)

[kylemcdonald]

very cool! thanks for the contribution :) next step is just plugging this into FaceTracker ;)

i'd like to integrate template matching into ofxCv at some point, it looks like using the ObjectFinder approach might be the way to go.

[crankdaworld]

@kylemcdonald I tried to pass the faces detected by finder.findHaarObjects to tracker.update method before reading this post. Doesnot seems to work . Where method has to be replaced in Face Tracker so that it uses Haar Object detector instead of default.

[antimodular]

did anyone succeed in tracking multiple faces?

[kylemcdonald]

@antimodular if this is something you are interested in implementing we could do a video chat some time and i can provide hints.

[antimodular]

oh yes we are interested. at this point went back to using ofxCv::ObjectFinder for multiple faces and put all that stuff in a separate thread. i even was able to run this with 5 separate usb webcam streams.

but nonetheless using the ofxFaceTracker would be better.

[antimodular]

if you have time it would be great to take a look at the multi face support. thanks,stephan

[kylemcdonald]

first you could create an ofxFaceTrackerMulti class. it inherits only from ofThread. the setup function would have an integer that says the maximum number of faces you want to track. inside setup you create an vector called trackers.

when you call ofxFaceTrackerMulti::update(image), it does a few things:

1. make a copy of image called buffer
2. trackers[0].update(buffer);
3. check if trackers[0] is found (this is old data, but it's ok). if not, return
4. draw a black square into buffer over the bounding box of the trackers[0] face (drawing this square can be done with opencv)
5. trackers[1].update(buffer)
6. check if trackers[1] is found (this is old data, but it's ok). if not, return
7. draw a black square into buffer over the bounding box of the trackers[1] face
8. trackers[2].update(buffer) ...

this continues for all the trackers.

then to get the tracked data, you can loop over the trackers with vector& ofxFaceTrackerMulti::getTrackers()

[antimodular]

i was successful thanks to your instructions. here my code: https://gist.github.com/antimodular/5eb36d6663614a40ed2e

it works now threaded and non-threaded

#define USE_THREADED

#ifdef USE_THREADED
#include "ofxFaceTrackerThreaded.h"
#else
#include "ofxFaceTracker.h"
#endif

next step is to make it more elegant in the cases when faces get lost. because right now if face 2 gets lost face 3 becomes face 2 and needs re-initialization.

[pikilipita]

Thanks for sharing your code antimodular, it works, but detects all faces at the same location, may you know why this is happening?

does you code draw a black square as suggested by kylemcdonald ?

[pikilipita]

Never mind, the issue was from my side. Sorry for that.