IJBC Evaluation metadata

[ksachdeva]

Hi,

First of all you guys are doing excellent work. Many thanks for this great work.

I was looking specifically for IJBC evaluation and that brought me to your repository.

I see some information here https://github.com/deepinsight/insightface/tree/master/Evaluation/IJB

and specifically looked at ijbc_face_tid_mid.txt which contains 469,375 entries (corresponds to 469,375 images). I got the specification and protocol files from here http://nigos.nist.gov:8080/facechallenges/IJB-C_Protocols_update.tar.gz which has bunch of CSV and meta data files. However, I could not find ijbc_face_tid_mid.txt or ijbc_name_5pts_score.txt in that bundle.

The images are to be dowloaded separately though after filling a form at NIST but your repository also provides the images (the loose_crop folder).

I am bit puzzled and would appreciate if you could help answer these questions -

a) Did you create/curate ijbc_face_tid_mid.txt and ijbc_name_5pts_score.txt files ? b) The images in loose_crop folder are the ones to be obtained after filling the form at NIST web site or these images were collected by you.

Many thanks

[jiankangdeng]

Thanks for your interest in our work. (1) ijbc_face_tid_mid.txt is from https://github.com/ox-vgg/vgg_face2/tree/master/standard_evaluation ijbc_name_5pts_score.txt is created by ourselves. (Our own face detector, which will be released recently)

(2) The loose_crop images are also from VGG group, https://github.com/ox-vgg/vgg_face2/tree/master/standard_evaluation as we follow their setting to test the performance on IJB. The loose crops may be different from the original images.

[ksachdeva]

Thanks @jiankangdeng for the clarifications.

[darshangera]

Hello Sir,

I am not able to find 469,375 images in IJB-C dataset. I have downloaded the datasets from NIST but I don't know which images to use for 1-1 face verification. I am able to see only 138836 (images +frames) in the downloaded set. Can you please help me in this? You have mentioned that you are using loose crop from VGG group. Are these different from what is given by NIST? Please can you help me in evaluating 1-1 face verification on IJB-B and C datasets? Thanking you

[matteo-pennisi]

@darshangera Have you solved? I am also trying to use IJB-B as a benchmark and I'm having some troubles.

[golunovas]

Thanks for your interest in our work. (1) ijbc_face_tid_mid.txt is from https://github.com/ox-vgg/vgg_face2/tree/master/standard_evaluation ijbc_name_5pts_score.txt is created by ourselves. (Our own face detector, which will be released recently)

(2) The loose_crop images are also from VGG group, https://github.com/ox-vgg/vgg_face2/tree/master/standard_evaluation as we follow their setting to test the performance on IJB. The loose crops may be different from the original images.

@jiankangdeng Were you referring to Retinaface detector?

[Edwardmark]

Thanks for your interest in our work. (1) ijbc_face_tid_mid.txt is from https://github.com/ox-vgg/vgg_face2/tree/master/standard_evaluation ijbc_name_5pts_score.txt is created by ourselves. (Our own face detector, which will be released recently)

(2) The loose_crop images are also from VGG group, https://github.com/ox-vgg/vgg_face2/tree/master/standard_evaluation as we follow their setting to test the performance on IJB. The loose crops may be different from the original images.

@jiankangdeng Hi, in these links, is IJBB face_tid_mid.txt, not ijbc, could you please give out an ijbc meta data? Thanks.

[whkwls2653]

I will leave comments for people who got stuck to use IJB-C, especially IJB-C 1:1 verification like me. I have confused about 'tid_mid.txt' and 'loose crop' After few pieces of research, Finally, I got.

---

After applying and getting IJB-C data set from the official page( https://www.nist.gov/itl/iad/ig/ijb-c-dataset-request-form), You will get several meta files. Among them, 'test1/enroll_templates.csv', 'test1/verif_templates.csv','test1/match.csv','ijbc_metadata_with_age.csv' are thing you will use from official dataset.

Now, you need 'tid_mid.txt', 'loose_crop', '5pts score.txt' you can download 'IJBC_tid_mid.txt', '5pts score.txt' at (https://github.com/deepinsight/insightface/tree/master/evaluation/IJB) +'mid' is the synonym of SIGHTING_ID in 'ijbc_metadata_with_age.csv (This fact is what I'm searching for a week) +you can use your own detector for 5pts score.txt. pls refer to MTCNN align code. (https://github.com/urbaneman/Face_crop_align_mtcnn) +'name_5pts_score' contains [filenames, 5landmarks coordinates, faceness score]

loos_crop is a sub dataset of IJB-C that is sorted and named according to 'tid_mid.txt'. I recommend using the bounding box coordinate in 'ijbc_metadata_with_age.csv' to crop images. ( you can use /IJB-C/utilities/ijbc_crop/crop.py but you need to modify and debug the code.)

---

I hope this helps you to research Face Recognition :)

[praneet195]

Can someone please create a step by step procedure for the 1:1 and 1:N evaluation of IJB-C with the new csv files ?

[CMakey]

I will leave comments for people who got stuck to use IJB-C, especially IJB-C 1:1 verification like me. I have confused about 'tid_mid.txt' and 'loose crop' After few pieces of research, Finally, I got.

After applying and getting IJB-C data set from the official page( https://www.nist.gov/itl/iad/ig/ijb-c-dataset-request-form), You will get several meta files. Among them, 'test1/enroll_templates.csv', 'test1/verif_templates.csv','test1/match.csv','ijbc_metadata_with_age.csv' are thing you will use from official dataset.

Now, you need 'tid_mid.txt', 'loose_crop', '5pts score.txt' you can download 'IJBC_tid_mid.txt', '5pts score.txt' at (https://github.com/deepinsight/insightface/tree/master/evaluation/IJB) +'mid' is the synonym of SIGHTING_ID in 'ijbc_metadata_with_age.csv (This fact is what I'm searching for a week) +you can use your own detector for 5pts score.txt. pls refer to MTCNN align code. (https://github.com/urbaneman/Face_crop_align_mtcnn) +'name_5pts_score' contains [filenames, 5landmarks coordinates, faceness score]

loos_crop is a sub dataset of IJB-C that is sorted and named according to 'tid_mid.txt'. I recommend using the bounding box coordinate in 'ijbc_metadata_with_age.csv' to crop images. ( you can use /IJB-C/utilities/ijbc_crop/crop.py but you need to modify and debug the code.)

I hope this helps you to research Face Recognition :)

sorry to bother u. IJBC had been discounted distribution , what else should i do now... .