cyrildiagne / ar-facedoodle

Doodle AR masks right from the browser with tensorflow.js
MIT License
545 stars 89 forks source link

Other than face ? #2

Open wardhanster opened 4 years ago

wardhanster commented 4 years ago

This is awesome, can this demo be replaced by something other than face? Let's say a cocacola can or something else

cyrildiagne commented 4 years ago

Hi,

Well yes and no ^^

In short terms, no, because I initially focused on the idea of "Drawing in 3D". So this demo uses a machine learning model that has been trained to recognise the 3D mesh of faces only: FaceMesh (paper).

Now, it would be possible to branch out this logic and draw in 2D only, in which cases it would be possible to use something like the Coco-SSD object detection model to detect a broader variety of elements such as bottles, plants, chairs..etc. But there would probably be little code left from this repo. So it would probably best to start from scratch and simply draw in canvas rather than with three.js.

There is also the Handpose model that now works with Tensorflow.JS and would be great for some puppet-like experiments.