search

WebAR-rocks / WebAR.rocks.face

WebAR.rocks face detection and tracking JavaScript library
https://webar.rocks
Other
84 stars 27 forks source link

texture mask different proportions #56

Open bartatgithub opened 1 year ago

bartatgithub commented 1 year ago

Hello

First of all, thanks for these incredible AR applications for the web!

For some time now I've been trying to use a longer texture, drawing of a 'dino' head in your makeupTexture. But this one obviously has different proportions, longer and the eyes are higher, than a human head and hence I don't get this working.

Currently, I keep seeing only a black mask with the normal human proportion but the 'dino' mask does not appear.

Can you explain to me the steps I need to take to get this working? Where I need to make adjustments in the code to use a mask texture with different proportion?

Thank you!

xavierjs commented 1 year ago

Hi @bartatgithub

Thank you for your interest in WebAR.rocks.face. I need to know more about what you mean by a "longer texture". Do you mean that you changed the aspect ratio of the image?

If the rendering is all black, it is likely because your new texture is not PoT (power of two). The makeupTexture demo is not a 3D demo, it is more 2D. The points of the mask are placed in 2D to match the face landmarks. It can only work with makeup of face painting, or something really stuck to the skin of the face.

If you need to place a 3D deformable mask, you need to use a textured 3D dino mask, rigged with a skeleton. Then the skeleton will follow the face landmarks thanx to some helpers. This demo is a good start: https://webar.rocks/demos/face/demos/autobones/ The source code is provided in this repo: https://github.com/WebAR-rocks/WebAR.rocks.face/tree/master/demos/autobones

Bones movement can be amplified compared to face landmarks displacement, it can be useful to animate the dino's beak for example. If you need some help, I offer development services, you can contact me at contact_at_webar.rocks

bartatgithub commented 1 year ago

Hi

Sorry, I'm sure I had replied from my smartphone but was from another country in the mountains so apparently didn't get through.

Yes, the aspect ratio of the dinosaur drawing is longer. Longer head with more distance between mouth and eyes. The dinosaur drawing: drawing

I also thought about your '3D deformable mask' version but the artist's dino drawing is 2D and he also wants to keep this 2D but with real-time movement.

If I get this dino drawn mask 'pasted' onto the face in the right proportions and adjust the position tracking of the mouth and eyes, can this work the same as the 'make-up' mask?

Can you direct me to where in the code I can remap the position tracking of the mouth and eyes to?

I am doing this for free for a friend artist, so there is currently no budget for development services. For me, it's an exercise at the same time because of my desire to delve into new website technologies.

Thanks for any help!