Testing on face verification

[anguoyang]

I have tested with some image pairs which belong to the same person, and uploaded one of the comparison result screenshot: 1. https://imgur.com/a/f3YDk the similarity between 2 person is about 51% based on the L2 distance --->similarity conversion: https://stats.stackexchange.com/questions/158279/how-i-can-convert-distance-euclidean-to-similarity-score 2. https://imgur.com/a/6dzRZ

[anguoyang]

maybe it is because that the ms-celeb data is dirty?

[thuhuwei]

Firstly, the similarity score cannot be computed with this simple method (a similarity score 1.0 only can be achieved when the distance value is 0.0) in face recognition, a better solution is to test lots of labeled pairs and analysis their results (or you can get their relationship based on our ROC curve on LFW). We provide thresholds only for academic comparison.

Secondly, our model is trained on celeb data, and it might not guarantee satisfied results on these images.

[anguoyang]

@thuhuwei Thank you for your reply, I have tried several open source projects, the result is similar, and I have tried to test with typical images in real environment, I am really curious about how those commercial libraries got so high score, is it possible to improve the open source projects to this level?

[huangyangyu]

As Prof. Hu said, how to acquire good similarity which range from 0 to 1 is not a easy problem. Different training method may have different way to compute the score. In our experience, 0.9385 which shown in given figure is a good result for the positive pair. Thank you for verifying our method.

[anguoyang]

I have also read some papers, the performance should not be so different, maybe they have huge training dataset

[anguoyang]

@huangyangyu how to translate your L2 distance into similarity?

[thuhuwei]

I'm sorry that you didn't catch the point of my answer. In fact, 0.93 is a good result for a positive pair, it means almost a 100% similarity score on LFW or YTF test sets.

[anguoyang]

Yes, they belong to the same person which captured in surveillance projects using IP camera, and selected/checked by testing team carefully, I am sure about it.

[anguoyang]

each pair contains 1 id photo(read from id card) and 1 image captured in surveillance camera, which belong to the same person, I have tested for about 30 pairs

[thuhuwei]

您应该能看懂中文。相似度是不能通过那个简单运算得到的。如果你想知道转换关系，可以根据我们提供的计算LFW和YTF的ROC数据，大概分析出距离和相似度之间计算公式（是一个非线性的转换）。根据我们在LFW和YTF上的测试情况，0.93的距离事实上差不多意味着100%的相似度。谢谢！

[anguoyang]

@thuhuwei 非常感谢！另外，这个非线性转换，是否通过类似于非线性回归这种方式获得？

[thuhuwei]

计算的方法有很多种（基于统计的、回归的等等），你研究一下ROC曲线的含义，就大概知道怎么做了

[anguoyang]

@thuhuwei 好的，多谢