search

oscarwestberg / Face-Tracking-Maya

A plugin for maya tracking a face using markers and animating a 3D face-mesh based on marker positions. A C++ program using OpenCV will fetch data from the webcam, the program will then use image analysis to find tracking points. These points will be streamed as raw data to Maya using its command port which then modifies the geometry of a 3D model of a face.
65 stars 17 forks source link

How usable is this in practice? Seems to be extremely buggy. #4

Open Githomie opened 5 years ago

Githomie commented 5 years ago

Hi, this is a wonderful project. I'm trying to get it working so I can interface with Maya.

Successfully built it on Win10 with a little elbow grease.

Had to screw around a lot with lighting and webcam drivers (Logitech c920) to adjust the image so it would pick up the markers (medical tape with highlighter on it).

Seems to only work in pitch black with a light directly on your face to whiten it. Also, note for anyone else trying this - recommended that you color sample your marker in the image to get the HSV value and recompile with that.

Finally got everything to work so that it detects the markers. HOWEVER - while they appear (relatively) stable in the image, the control points jump all over the place in Maya, pulling the face in bizarre distorted manner. Occasionally, I can calm it down enough that it'll be stable for a second or two.

So it would really help if you could post the lighting conditions you used, markers, etc..

It also seems to crash after a few minutes (once it's connected to Maya). Is this normal or a bug associated bad marker detection.

Would love to get this working, as most decent facial capture seems to be prohibitively expensive for me.

Thanks!

oscarwestberg commented 5 years ago

How usable is it in practise? I'd say hardly at all. This was done as a school project with very limited time so just about every part of this is made to work for our use case but completely untested for everything else.

If you're serious about getting face tracking for Maya but don't want to pay for an expensive product your best bet is to write your own solution. Feel free to take as much inspiration as you want from this project.

Also the readme might not be updated with the correct number of markers or their position.. I'd love to make this into a solid project for others to use but as it stands this project isn't going anywhere.

Best of luck