Face detection algorithms

[Clorr]

Following an issue on main faceswap repo, I tried to benchmark misc face detection approaches.

Here is the code I used.

The test photo from the issue had a very inclined face, which was only detected by mmod_human_face_detector of dlib

import cv2
from PIL import Image
import face_recognition

# Load the jpg file into a numpy array
image = cv2.imread("src/test.jpg")

# Find all the faces in the image using the default HOG-based model.
# This method is fairly accurate, but not as accurate as the CNN model and not GPU accelerated.
# See also: find_faces_in_picture_cnn.py
face_locations = face_recognition.face_locations(image)

print("I found {} face(s) in this photograph.".format(len(face_locations)))

for face_location in face_locations:

    # Print the location of each face in this image
    top, right, bottom, left = face_location
    print("A face is located at pixel location Top: {}, Left: {}, Bottom: {}, Right: {}".format(top, left, bottom, right))

    # You can access the actual face itself like this:
    face_image = image[top:bottom, left:right]
    pil_image = Image.fromarray(face_image)
    pil_image.show()

########################################################################

import cv2
import numpy

# Give right path to the xml file or put it directly in current folder
face_cascade = cv2.CascadeClassifier('haarcascade_frontalface_default.xml')

#Add : cv.EqualizeHist(image, image) ?
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)

# Detect faces in the image
faces = face_cascade.detectMultiScale(
    gray,
    scaleFactor=1.1,
    minNeighbors=5,
    minSize=(30, 30),
    flags = cv2.CASCADE_SCALE_IMAGE
)
print("I found {} face(s) in this photograph with haarcascade.".format(len(face_locations)))

for (x,y,w,h) in faces:
    face_image = image[y: y + h, x: x + w]
    pil_image = Image.fromarray(face_image)
    pil_image.show()

########################################################################

import sys
import dlib
from skimage import io

detector = dlib.get_frontal_face_detector()

dets = detector(image, 1)
print("Number of faces detected: {}".format(len(dets)))
for i, d in enumerate(dets):
    print("Detection {}: Left: {} Top: {} Right: {} Bottom: {}".format(
        i, d.left(), d.top(), d.right(), d.bottom()))

########################################################################

import sys
import dlib
from skimage import io

detector = dlib.cnn_face_detection_model_v1("mmod_human_face_detector.dat")

dets = detector(image, 1)
print("Number of faces detected: {}".format(len(dets)))
for i, d in enumerate(dets):
    print("Detection {}: Left: {} Top: {} Right: {} Bottom: {}".format(
        i, d.rect.left(), d.rect.top(), d.rect.right(), d.rect.bottom()))

[Jack29913]

Do we have this one as extract plugin?

[Clorr]

Hmm, this is not just a matter of extracting, the face detection is used in extract AND in convert. So if we add it on extract, we have to add it in convert also, so it can't be a "plugin", it is more internal.

But it is always possible to add it in lib/faces_detect.py, with a param to activate it

[Jack29913]

Yeah let's do it like that. Also in the config file