search

deepfakes / faceswap-model

Tweaking the generative model
147 stars 133 forks source link

Feasability - use for facial tracking/paint work #9

Closed tk421storm closed 6 years ago

tk421storm commented 6 years ago

Hey all -

I work in the VFX industry, and I was curious - how feasible would code similar to this be for tracking an artist's work (on a single frame) onto a face?

For example, we are often asked to paint in/out elements on an actors face (add scars, remove acne). From my understanding, the algorithm takes multiple images of each actor and creates a parity model to be applied. Would there a way to then do a paint job on a single frame, and apply that paint through the tracked model?

Thanks!

-Mike

shaoanlu commented 6 years ago

I think its possible for the model to learn adding paint jobs on faces, but not at production-level quality. Its challenging for auto-encoders (and GANs as well) to produce high resolution images, say, 1024x768, thus much more challenging for videos.

As for this project, you can try to put faces w/ scars into folder 'faceA' and faces w/o scars into folder faceB, then see if the results meet your expectation.

tk421storm commented 6 years ago

Thanks for you help! It sounds like we'd still have to do the paint on multiple source images, though - probably not a great solution.

It's neat, anyway, and I'm sure I'll find some way to use it.