frankier / skelshop

📺 📰 🧑‍💼 Toolkit for skeleton & face analysis of talking heads (e.g. news) videos 🧑‍💼 📰 📺
https://frankier.github.io/skelshop
MIT License
5 stars 1 forks source link

Add Insightface ARCFace as a face embedder #37

Open frankier opened 3 years ago

frankier commented 3 years ago

This is a tracking issue for different face embeddings. The main criteria what the cropping procedure is (is it compatible with dlib?) are how well it performs on different benchmarks and the size. LFW is compared first since it's the most common.

Name Dim LFW acc Ref
insightface ARCFace 512 99.83 1
FaceNet trained on VGGFace 2 512 99.63 1
vgg-face embedding loss 1024 99.13 1
dlib 128 98.4/99.38 1 2

Better evaluation: https://www.cs.tau.ac.il/~wolf/ytfaces/ (v. relevant here) https://ieeexplore.ieee.org/document/8575544

frankier commented 3 years ago

Looks like Insightface ARCFace is the only one worth adding:

https://github.com/nizhib/pytorch-insightface https://github.com/deepinsight/insightface

Keypoints are here: https://github.com/deepinsight/insightface/blob/5b40f4bfce7e2c9d10bb6328c4fed33e9d76c9de/python-package/insightface/utils/face_align.py#L31

frankier commented 3 years ago

Note that ARCFace probably uses cosine distance, unlike dlib so this needs to be taken account of. (Need to double check this.)