Face detector performance is low on small images.

[jcheong0428]

Testing on 48x48 fer2013 led to undetected faces...

[mperreir]

This is because py-feat tries to detect the faces in the image prior to detecting the emotions. In fer2013, since the images are already cropped in order to only include the face area, the face detection algorithm fails. You need to modify detector.py so as to bypass the face detection step and use the whole face area instead detected_faces = [[[0,0,47,47,1]]]. Of course this is only valid for fer2013 images. A cleaner solution would be to modify detector.py so that you have the option to bypass the face_detection step.

[ljchang]

Good point @mperreir. @ejolly as we continue to finish up https://github.com/cosanlab/py-feat/pull/133, we may want to now allow specific modules to be turned off in the main image/video detector methods.

I agree with @mperreir , that the easiest solution is to just write a few extra lines of code in a script with the lower level detectors to accommodate this type of data.

For example,

from torchvision.io import read_image
from torchvision.transforms import Compose, Pad
from feat.utils.io import get_test_data_path
from feat.utils.image_operations import extract_face_from_bbox, convert_image_to_tensor
from feat.plotting import draw_lineface
from feat import Detector
import matplotlib.pyplot as plt
import os

# Extract Faces from Image
multi_face_img_path = os.path.join(get_test_data_path(), "multi_face.jpg")
img = read_image(multi_face_img_path)
faces = detector_cpu.detect_faces(img)
face_imgs = extract_face_from_bbox(convert_image_to_tensor(img), faces)[0]

# Initialize detector
detector = Detector(device='cpu', landmark_model='pfld')

f,ax = plt.subplots(ncols=len(face_imgs), figsize=(15,3))
for i,face_img in enumerate(face_imgs):

    # Add padding to image
    padding = 20
    padded_face = Compose([Pad(padding, fill=(128))])(convert_image_to_tensor(face_img))

    # Create a new bbox for the landmark detector based on the shape of the face image 
    new_face_bbox = [[[padding, padding, face_img.shape[1]+padding, face_img.shape[1]+padding, 0.99]]]
    landmarks = detector.detect_landmarks(padded_face, new_face_bbox)

    # Plot Image & Landmarks
    ax[i].imshow(padded_face.squeeze().permute(1,2,0).numpy())
    ax[i].axes.get_xaxis().set_visible(False)
    ax[i].axes.get_yaxis().set_visible(False)

    currx = landmarks[0][0][:,0]
    curry = landmarks[0][0][:,1]
    draw_lineface(
        currx, curry, ax=ax[i], color='w', linewidth=3
    )

@ejolly, We could add this as an example in the tutorials.