About the performance of MobileFaceNet on IJB dataset

[mawj111]

Recently, I have used ijb_evals.py to test mobilefacenet based on arcface. The train dataset is CASIA-WebFace and the performance is as follows，and I wonder if the results are reasonable because N1D1F0 TAR@FAR=0.01% is only 0.3. Is the truth similar to my result?

	1e-06	1e-05	0.0001	0.001	0.01	0.1	AUC
mobilefacenet_arcface_IJBB_11_N1D1F1	0.0109056	0.0493671	0.303213	0.72853	0.904576	0.975365	0.989172
mobilefacenet_arcface_IJBB_11_N1D1F0	0.0113924	0.0507303	0.3	0.718111	0.901071	0.973905	0.988566
mobilefacenet_arcface_IJBB_11_N1D0F1	0.00769231	0.0395326	0.237293	0.6852	0.896203	0.974586	0.988449
mobilefacenet_arcface_IJBB_11_N1D0F0	0.00788705	0.040409	0.229503	0.674002	0.891237	0.972152	0.987863
mobilefacenet_arcface_IJBB_11_N0D1F1	0.011295	0.0539435	0.240117	0.696981	0.900682	0.97371	0.988731
mobilefacenet_arcface_IJBB_11_N0D1F0	0.0130477	0.0487829	0.237001	0.681305	0.897468	0.972541	0.988158
mobilefacenet_arcface_IJBB_11_N0D0F1	0.00895813	0.0380721	0.184907	0.641285	0.891431	0.971081	0.98786
mobilefacenet_arcface_IJBB_11_N0D0F0	0.0111003	0.0370983	0.182376	0.62259	0.886855	0.97001	0.987306

	1e-06	1e-05	0.0001	0.001	0.01	0.1	AUC
mobilefacenet_arcface_IJBC_11_N1D1F1	0.0172317	0.0596206	0.315795	0.73544	0.914557	0.978013	0.989999
mobilefacenet_arcface_IJBC_11_N1D1F0	0.0170783	0.0696426	0.322953	0.729918	0.909802	0.975661	0.989554
mobilefacenet_arcface_IJBC_11_N1D0F1	0.0143171	0.0478601	0.255203	0.697909	0.905763	0.976632	0.989416
mobilefacenet_arcface_IJBC_11_N1D0F0	0.0141637	0.0542517	0.263793	0.692591	0.902132	0.97428	0.988943
mobilefacenet_arcface_IJBC_11_N0D1F1	0.0193793	0.0634044	0.259754	0.69208	0.910927	0.976172	0.989532
mobilefacenet_arcface_IJBC_11_N0D1F0	0.0183566	0.0713811	0.263384	0.682773	0.907194	0.974843	0.989108
mobilefacenet_arcface_IJBC_11_N0D0F1	0.0125275	0.0494963	0.210359	0.636703	0.901621	0.974792	0.98885
mobilefacenet_arcface_IJBC_11_N0D0F0	0.0133456	0.0496497	0.21849	0.631181	0.897633	0.973053	0.988405

[mawj111]

I‘m sorry that I made a mistake. The above performance is based on RGB images, but my model is trained by BGR images. After converting RGB to BGR, the performance is as follows. N1D1F0 TAR@FAR=0.01% is 0.571373, and I wonder if the results are close to the truth because I hardly see anyone using IJB dataset to test mobilefacenet.

	1e-06	1e-05	0.0001	0.001	0.01	0.1	AUC
mobilefacenet_arcface_IJBB_11_N1D1F1	0.0201558	0.135735	0.57887	0.828238	0.931256	0.979649	0.991946
mobilefacenet_arcface_IJBB_11_N1D1F0	0.0191821	0.142843	0.571373	0.823174	0.92853	0.978384	0.991585
mobilefacenet_arcface_IJBB_11_N1D0F1	0.0196689	0.116358	0.494547	0.811392	0.925998	0.978481	0.991413
mobilefacenet_arcface_IJBB_11_N1D0F0	0.01889	0.119864	0.493087	0.807108	0.92259	0.97702	0.99106
mobilefacenet_arcface_IJBB_11_N0D1F1	0.0245375	0.139338	0.510613	0.822298	0.931451	0.978773	0.991639
mobilefacenet_arcface_IJBB_11_N0D1F0	0.0205453	0.144693	0.503797	0.817722	0.927653	0.977605	0.991304
mobilefacenet_arcface_IJBB_11_N0D0F1	0.0176241	0.114021	0.433593	0.802921	0.924343	0.97702	0.990989
mobilefacenet_arcface_IJBB_11_N0D0F0	0.0182084	0.117332	0.427848	0.797079	0.920837	0.975657	0.990669

	1e-06	1e-05	0.0001	0.001	0.01	0.1	AUC
mobilefacenet_arcface_IJBC_11_N1D1F1	0.0388608	0.21895	0.624073	0.858414	0.94222	0.982359	0.992794
mobilefacenet_arcface_IJBC_11_N1D1F0	0.0359462	0.224523	0.619931	0.853096	0.939357	0.98149	0.992609
mobilefacenet_arcface_IJBC_11_N1D0F1	0.0307818	0.167561	0.549522	0.840262	0.937874	0.981592	0.992444
mobilefacenet_arcface_IJBC_11_N1D0F0	0.0308329	0.179936	0.549062	0.835864	0.935164	0.980467	0.992249
mobilefacenet_arcface_IJBC_11_N0D1F1	0.0413151	0.206934	0.553715	0.850386	0.940737	0.981541	0.992512
mobilefacenet_arcface_IJBC_11_N0D1F0	0.0425934	0.227029	0.556578	0.845835	0.937618	0.980723	0.992334
mobilefacenet_arcface_IJBC_11_N0D0F1	0.0263333	0.152068	0.487754	0.829115	0.935931	0.980314	0.99212
mobilefacenet_arcface_IJBC_11_N0D0F0	0.0295546	0.163113	0.479624	0.824462	0.933221	0.979394	0.991928

[hairenaa]

what is N1D1F1

[kormalev]

@hairenaa

what is N1D1F1

Considering embeddings both for the original and Flipped (F=1) image, using Normalized (N=1) embeddings, weighing embeddings by Detector (D=1) score.

See here and here