search

jeeliz / jeelizFaceFilter

Javascript/WebGL lightweight face tracking library designed for augmented reality webcam filters. Features : multiple faces detection, rotation, mouth opening. Various integration examples are provided (Three.js, Babylon.js, FaceSwap, Canvas2D, CSS3D...).
https://jeeliz.com
Apache License 2.0
2.68k stars 533 forks source link

Detect eyes, nose and mouth precisely. #1

Closed jointdev closed 6 years ago

jointdev commented 6 years ago

Congratulations for this incredible library!

Can I get precise coordinates of eyes, mouth and nose?

Let me explain what I want to do ... I'm not a blander/threeJs professional and I want to use CSS3 capabilities to position objects on camera using coordinates delivery from jellizFaceFilter library. This is possible?

xavierjs commented 6 years ago

Hello,

Thank you for your interest about JeelizFaceFilter API :) Unfortunately the detection of the face is a global detection. We do not detect specific keypoints like BRFv4 for example (https://github.com/Tastenkunst/brfv4_javascript_examples).

We are going to release an upgrade where the mouth opening coefficient is detected.

But the current head detection frame is centered on the middle of the eyes so you can retrieve the position of the eyes quite accurately. Idem for the nose. But for the mouth I think there will be more variability because there are people with a lot of space between the mouth and the nose, and other whose mouth is almost stitched to the nose.

I will put the dev of a demonstration using canvas2D or CSS3 on my pipeline and keep you informed.

jointdev commented 6 years ago

Great! I already tested the brfv4 library before but I prefer your solution, including I can't use the brfv4 in a commercial app and the licence is expensive (in brazilian currency).

I'm following the "luffys hat" tutorial that is a perfect start point for my project.

I will share my project in github when I finish him.

Thank you very much.

xavierjs commented 6 years ago

Hi, We have added the CSS3D support for easier integration. The link to the boilerplate and another CSS3D example are included from the README. I tested it on Ipad, on android and desktop.

nitish24p commented 6 years ago

Is it possible to only color certain parts of the face, like the eyes or the lips and then the colored portion moves as the person smiles etc

xavierjs commented 6 years ago

this is not possible bc the detection is global