Closed YongtaoGe closed 6 years ago
def evaluate_with_no_cv(emb_array, actual_issame): thresholds = np.arange(-4, 4, 0.01) embeddings1 = emb_array[0::2] embeddings2 = emb_array[1::2]
nrof_thresholds = len(thresholds)
accuracys = np.zeros((nrof_thresholds))
diff = np.subtract(embeddings1, embeddings2)
dist = -np.sum(np.square(diff),1)
for threshold_idx, threshold in enumerate(thresholds):
_, _, accuracys[threshold_idx] = facenet.calculate_accuracy(threshold, dist, actual_issame)
best_acc = np.max(accuracys)
best_thre = thresholds[np.argmax(accuracys)]
return best_acc,best_thre
Well, I have done a litte change to the code so that it can calulate the cos similarity between embeddings when evaluating.
My reference is https://github.com/wy1iu/sphereface/tree/master/test/code/evaluation.m.
@geyongtao @GangqiangZhao Hey guys, sorry for late reply! I notice the problem when evaluating on LFW. It was caused by np.greater() function in function "calculate_accuracy" from facenet.py. Just modify it into np.less(). The reason I use np.greater() is that cosine distance was used to calculate similarity. But actually in the repo, euclidean distance was used, so np.less() is right. I already corrected it, it should works fine now.
@geyongtao
Is that ok just using the euclidean-based distance to evaluate?
Yes, of course. Euclidean distance and cosine distance are actually same when the length of vector are both 1.
Using the euclidean-based algorithm to evaluate is logically feasible, so what's problem for the fixed accuracy? Is there anything I forget to configure before training?
Sorry about this. It's my problem. The reason is on top of this reply, just simply change np.greater() into np.less() (which I already corrected it in this repo), because I used cos-distance to compare before, (now is euclidean distance).
Thanks for pointing this out : )
Hi, @Joker316701882. I use the CASIA-Webface as training set and the LFW dataset as validation set. However, during the training process, the accuracy on LFW dataset is always 100% and the selected threshold is ok. I read the source code print the 'acc' in facenet.calculate_accuracy find that the Acc gradually increases from 0 to 1. In train.evaluate_with_no_cv, you do those op: best_acc = np.max(accuracys) best_thre = thresholds[np.argmax(accuracys)] return best_acc,best_thre so the acc is always 100%. my problem is how to fix it and it is ok to get thres without cross-validation?
@weihehdc
how to fix it?
There is no threshold can lead acc to 1 (theoretically upper-bound acc is 99.86%). There must be something wrong with your implementation (or mine).
without cross-validation?
It's also theoretically better to use cross-validation. But there is almost no performance difference on lfw (on other test set there might be, because lfw here is validation set).
Hi, @Joker316701882. I use the CASIA-Webface as training set and the LFW dataset as validation set. However, during the training process, the accuracy on LFW dataset is always 50% and the selected threshold is always 0. To find the problem, I read the source code and find the euclidean distance is used to calculate the distance between the two embedded features. As I know, AM-Softmax is a angular-based face recognition algorithm. My problem is: