search

deepfakes / faceswap-model

Tweaking the generative model
147 stars 133 forks source link

Better face detection algorithm #2

Closed Clorr closed 6 years ago

Clorr commented 6 years ago

Face detection is still perfectible. suggest better alternatives if you have some idea

Clorr commented 6 years ago

I gave a try at https://github.com/ageitgey/face_recognition

I thought it was mostly a repackaging of dlib, but it actually solves the 2 issues I raised on faceswap-playground!

@Ganonmaster can you give it a try and see if it is better for you. I pushed it in my own repo https://github.com/Clorr/faceswap

(It is also pushed with a Dockerfile rework and reqs should be cleaned before PR-ing it...)

Clorr commented 6 years ago

Ow sh*t, it's been pushed to the upstream repo.... I'll check how I did that, and revert

(It seems I haved messed things with Github Desktop, but I don't understand why a sync with my account pushes things to the upstream repo...)

Ganonmaster commented 6 years ago

Very exciting, I'll be taking a look tomorrow. I'm bouncing between two workstations; don't have the processing power to do anything substantial other than some basic refactoring right now. Very promising though!

Clorr commented 6 years ago

Yeah I had a look at the code and was thinking it wouldn't change much, but actually, it did very well. When installing, it fetches its own models, and that's maybe where the magic comes from ;-)

Clorr commented 6 years ago

Also I realize my commit from yesterday with an aligner is also in the upstream...

I was not satisfied with it, have you seen a difference ? Do you want me to remove it, or is it ok for you ?

Ganonmaster commented 6 years ago

I've not seen an improvement for the aligner so far, it seem to not have had much impact.

However, I just ran the extract script on the Trump folder with the face_recognition lib, and the results seem to be so much better. Scanning the Trump dataset yielded roughly 120 faces out of 376 images, whereas the new lib managed to grab 310. Results for my Charlize Theron and Clinton pictures are similar. Going to try and test it with a trained model asap. 😃

Ganonmaster commented 6 years ago

Not sure if it's the aligner's fault, the test data, or the new face recognition, but some of these converted photos seem to be the proper size, while on others it seems to be enlarging the resulting faces. out234 out249 out218

Clorr commented 6 years ago

Yes it's an aligner issue I also encountered, I'll revert that...

But it's a great news for the recognition !

Clorr commented 6 years ago

Btw if you want to mute the aligner, it is in lib/faces_process.py . The old superpose is there as a fallback

deepfakes commented 6 years ago

Hi guys!

I had a review of the code, things go well I see ;-)

@Clorr I disabled the Aligner, but the rest seems fine @Ganonmaster I did a little modification on cli.py to preserve Python2 compatibility.

If you have ideas to move on to a full Python3 setup, it would be pretty cool. About folder structure, it is a bit messy for now, once we args parsing on all entrypoints, we should decide about a more standard/usable folder structure, and also, we will have to update the docs...

Keep up the good work guys ! It's moving on pretty well!

Ganonmaster commented 6 years ago

I'm doing pretty much all my testing on Python3 and would suggest making that the standard for this project going forward. "New" projects like this should strive to not depend on legacy software, as it only delays the inevitable and makes transitioning more painful. That does not mean it can't be Python2 compatible, but I would strongly recommend developing for Python3 first and Python2 as a nice to have.

deepfakes commented 6 years ago

I totally agree we should move on to python3. The problem is the Dockerfile has been built with python2, and if user try the project with it, it will fail, so it's a bit sad. But, all in all, it is just a matter of migrating the Dockerfile...

Yutsa commented 6 years ago

On deepfakes subreddit there are lots of different versions of the scripts like this one for instance.

Is it better than what we have now ? Should we try to integrate it ?

Clorr commented 6 years ago

This is more a face alignment patch than a face detection one. But yes it would be nice to have it integrated. However the face alignment part relies on pytorch which is an annoying dependency (heavy and not available on windows).

So the seamless clone face merge part can be integrated if it is more performant than what we have. The face alignment part could be interesting but not as the official algorithm. So it can pushed in a branch if you want. Or even better, we could replace the face landmarks detection by something we already have in ‘face-recognition‘ module