The Face of Art: Landmark Detection and Geometric Style in Portraits

Code for the landmark detection framework described in The Face of Art: Landmark Detection and Geometric Style in Portraits (SIGGRAPH 2019)

_{^{Top: landmark detection results on artistic portraits with different styles allows to define the geometric style of an artist. Bottom: results of the style transfer of portraits using various artists' geometric style, including Amedeo Modigliani, Pablo Picasso, Margaret Keane, Fernand Léger, and Tsuguharu Foujita. Top right portrait is from 'Woman with Peanuts,' ©1962, Estate of Roy Lichtenstein.}}

Getting Started

Requirements

python
anaconda

Download

Model

download model weights from here.

Datasets

The datasets used for training and evaluating our model can be found here.
The Artistic-Faces dataset can be found here.
Training images with texture augmentation can be found here. before applying texture style transfer, the training images were cropped to the ground-truth face bounding-box with 25% margin. To crop training images, run the script crop_training_set.py.

our model expects the following directory structure of landmark detection datasets:

landmark_detection_datasets
├── training
├── test
├── challenging
├── common
├── full
├── crop_gt_margin_0.25 (cropped images of training set)
└── crop_gt_margin_0.25_ns (cropped images of training set + texture style transfer)

Install

Create a virtual environment and install the following:

opencv
menpo
menpofit
tensorflow-gpu

for python 2:

conda create -n foa_env python=2.7 anaconda
source activate foa_env
conda install -c menpo opencv
conda install -c menpo menpo
conda install -c menpo menpofit
pip install tensorflow-gpu

for python 3:

conda create -n foa_env python=3.5 anaconda
source activate foa_env
conda install -c menpo opencv
conda install -c menpo menpo
conda install -c menpo menpofit
pip3 install tensorflow-gpu

Clone repository:

git clone https://github.com/papulke/deep_face_heatmaps

Instructions

Training

To train the network you need to run train_heatmaps_network.py

example for training a model with texture augmentation (100% of images) and geometric augmentation (~70% of images):

python train_heatmaps_network.py --output_dir='test_artistic_aug' --augment_geom=True \
--augment_texture=True --p_texture=1. --p_geom=0.7

Testing

For using the detection framework to predict landmarks, run the script predict_landmarks.py

papulke / face-of-art

readme