Reproduce the results

[hahv]

Thank you for the work. I want to know how to reproduce the result in this report I was unable to achieve the provided performance on LFW data (lfw.bin from InsightFace data zoo->MS1M-ArcFace) For example, with the following model:

I obtained the results as follow:

lfw evaluation max accuracy: 0.998333. However, it should be 0.998667 as in the report.

Note that:

1. When I used lfw.bin (61.5 mb) provided in this repo release, the results will be the same as in the table
2. When I used lfw.bin (61.7 mb) provided in the InsightFace data zoo->MS1M-ArcFace, the result was lower as I mentioned before.

So, what are the differences between your provided lfw.bin @leondgarse and the corresponding file in InsightFace data zoo->MS1M-ArcFace? Does it account for the difference when reproducing the results?

[leondgarse]

Ya, you are right on the results. The 0.998667 one is from InsightFace data zoo->ms1m-retinaface, and yes, it's testing result is a bit higher than those lfw.bin from other datsets... Here are my test results:

MODEL_PATH="./checkpoints/TT_r100_max_pool_E_prelu_dr04_lr_01_l2_5e4_adaface_emb512_sgd_m09_bs512_ms1m_64_only_margin_SG_scale_true_bias_false_random_100_basic_model_latest.h5"
DATASET_PATH="/datasets/ms1m-retinaface-t1"
python3 evals.py -m ${MODEL_PATH} -t ${DATASET_PATH}/lfw.bin ${DATASET_PATH}/cfp_fp.bin ${DATASET_PATH}/agedb_30.bin 
# Evaluating lfw: 100%|███████████████████████████████████|  94/94 [00:47<00:00,  1.98it/s]
# >>>> lfw evaluation max accuracy: 0.998667, thresh: 0.240591, previous max accuracy: 0.000000
# >>>> Improved = 0.998667

# Evaluating cfp_fp: 100%|███████████████████████████████████|  110/110 [00:51<00:00,  2.15it/s]
# >>>> cfp_fp evaluation max accuracy: 0.992286, thresh: 0.181722, previous max accuracy: 0.000000
# >>>> Improved = 0.992286

# Evaluating agedb_30: 100%|███████████████████████████████████| 94/94 [00:43<00:00,  2.18it/s]
# >>>> agedb_30 evaluation max accuracy: 0.984333, thresh: 0.172596, previous max accuracy: 0.000000
# >>>> Improved = 0.984333

Source dataset	lfw max accuracy	cfp_fp max accuracy	agedb_30 max accuracy
MS1M-ArcFace	0.998333	0.990857	0.984500
faces_casia	0.998333	0.970143	0.984333
glint360k	0.998333	0.990857	0.984333
ms1m-retinaface	0.998667	0.992286	0.984333

Maybe just a difference in image encoding qulity when saving those bin files. You may just pick a same one comparing their accuracies.

[hahv]

I got it. Thank a lot 🙏