Transform image using facial features

[andreaseliasson]

Suggestion:

Using the facial features we can transform the image to make the face as centered as possible. This will help with faces that are turned in weird directions.

Aligned faces can now be downloaded from: http://www.edshare.soton.ac.uk/18368/ Put the csv file in the data directory.

[andreaseliasson]

The openface library can solve this problem with a specific method; however, it doesn't support python3.

We can use opencv (openface uses opencv behind the scenes) for the transformation, but this will require more code, including some computer vision logic.

[andreaseliasson]

How to use openface

It seems the easiest way to align the faces in the images is to use the openface library, and the easiest way to use openface is via a preconfigured docker image.

Before we start

Run the modified file src/data2img.py to split the images into training and test images. Pull the docker image which will have openface, dlib and python pre-installed:

docker pull bamos/openface
docker run -p 9000:9000 -p 8000:8000 -t -i bamos/openface /bin/bash
cd /root/openface

Use the -v flag to make your local folders available inside the docker image. On Mac I do:

docker run -v /Users:/host/Users -p 9000:9000 -p 8000:8000 -t -i bamos/openface /bin/bash
cd /root/openface

On Linux, I am not sure how... Now I can access my files: ls /host/Users/

Step 1.

Make a folder called ./training-images/ inside the openface folder.

mkdir training-images

Step 2.

Make a subfolder for each of the expressions. For example:

mkdir ./training-images/0/
mkdir ./training-images/1/

Step 3

Copy all local images of each expression into the correct sub-folders. For example:

cp -a xxxxxx/facial-expression-recognition/data/img/train/0/. ./training-images/0/

Step 4

Run the openface scripts from inside the openface root directory:

Pose detection and alignment:

./util/align-dlib.py ./training-images/ align outerEyesAndNose ./aligned-images/ --size 48

This will create a new ./aligned-images/ subfolder with an aligned version of each of the test images.

Now we can copy the contents of ./aligned-images/ back into the project data directory.

cp -a aligned-images/. xxxxx/facial-expression-recognition/data/img/train_aligned/

We should now have a new directory in our project with the aligned training images at /facial-expression-recognition/data/img/train_aligned/