Mouth tracker

[rafaelfaria]

Hi mate,

Sorry to bother you with all these questions, but have you ever manage to do any testing to do a mouth tracking?

I'm no nerd or great with math so I was wondering if you could help me out.

thanks

[wen-bing]

Sorry, either me not good at math. :(

2012/9/20 rafaelfaria notifications@github.com

Hi mate,

Sorry to bother you with all these questions, but have you ever manage to do any testing to do a mouth tracking?

I'm no nerd or great with math so I was wondering if you could help me out.

thanks

— Reply to this email directly or view it on GitHubhttps://github.com/auduno/headtrackr/issues/2.

[auduno]

That depends a bit on whether you just want to track the general position of the mouth, or if you want to track more precise movements of the mouth, like closed or open, etc. I haven't tried tracking the mouth myself though.

If you just want to track the general position, you could use feature detection to detect the mouth and maybe track it with camshift. For feature detection, I know it's possible to train new feature detectors in ccv.js (which I use in headtrackr library), but I haven't tried training new models myself, so don't know how hard it is. Take a look at this link for doing so: http://libccv.org/doc/doc-bbf/#how-to-train-my-own-detector

You might also look at haar.js, which is a similar feature detector, but which is based on openCV models. It seems like this already has a mouth detector model, so maybe it's easier to use this.

If you want to track more precise actions of the mouth, I don't think there currently exists any javascript libraries to do that, unfortunately.

[rafaelfaria]

Yeah, i had a look at haar.js but I think they use a different json for the face recognition. Is it something you might be able to help me, maybe create a "mouthtracker js"? lol.

I'm trying to work out the movement of the mouth to have more accuracy of the moviments "up" and "down" of the head, as in right now your script (which is the best on i could find) tracks the whole intensity of the head and turns out to not be that accurate for what I'm doing.

Thanks

[rafaelfaria]

I've just tried the haar.js cascade for the mouth and i get "Uncaught RangeError: Invalid array length". Obviously because they are different range of array. I'm kinda lost of what I can do. Sorry to bother you!

[auduno]

Yes,

haar.js uses a different format, so they're not interchangable unfortunately. I can have a look at whether tracking the mouth is at all feasible using my tracking mechanism over the weekend.

[rafaelfaria]

Hey mate,

if you manage to do this, you would save my life! :)

thank you very much!!!

[auduno]

Hi,

I tried it out, and I don't think it's feasible. The tracker that I use (camshift) won't be able to "lock" onto as small areas as the mouth, and the color of the mouth is not distinct enough from the rest of the face.

You can try it out yourself by selecting the area of the mouth in this example here: http://auduno.github.com/headtrackr/examples/camshift.html

So even if you manage to detect the mouth with haar.js, you won't be able to track it. :(

[rafaelfaria]

Hey Mate,

Thank you anyway, i will try to work with what I have. Great work thou. one of the best open source javascript for face recognition i've found so far.

Not sure if I understand what your example was suppose to do, it just show the camera but nothing happens.

Thanks Raf

On Mon, Oct 8, 2012 at 4:31 AM, Audun Mathias Øygard < notifications@github.com> wrote:

Hi,

I tried it out, and I don't think it's feasible. The tracker that I use (camshift) won't be able to "lock" onto as small areas as the mouth, and the color of the mouth is not distinct enough from the rest of the face.

You can try it out yourself by selecting the area of the mouth in this example here: http://auduno.github.com/headtrackr/examples/camshift.html

So even if you manage to detect the mouth with haar.js, you won't be able to track it. :(

— Reply to this email directly or view it on GitHubhttps://github.com/auduno/headtrackr/issues/2#issuecomment-9210199.

[auduno]

Hi,

sorry, I wasn't so clear. In the example, if you click and drag a square around an object on the video, you can select an area/object to track. So if you select a box around the mouth, you will see that it will start to track the whole face instead of the mouth. :/

-Audun