hbdat / cvpr20_LESA

Shared Attention for Multi-label Zero-shot Learning accepted @ CVPR20
MIT License
31 stars 6 forks source link

Assertion error during open-images training #3

Closed akshitac8 closed 3 years ago

akshitac8 commented 3 years ago

Hello Dat, The total length of the top unseen is 400 but it's 399 at L55 of the training file. It would be great if you could let me know the correct number.

Thanks in advance, Akshita

hbdat commented 3 years ago

Hi Akshita,

Thanks for carefully inspecting the code. The assertion should be 400 as there are 400 labels in the csv file. I have corrected this in my latest commit (could not recall why 399 appears there).

Best, Dat

akshitac8 commented 3 years ago

Thank you @hbdat

akshitac8 commented 3 years ago

@hbdat I also wanted to ask how many numbers of images do you have after masking for ZSL evaluation for open-images?

hbdat commented 3 years ago

Hi @akshitac8,

The number of testing samples, used to computed mAP scores for ZSL and GZSL, are different across different labels. This is because a sample could lack annotation for label ‘A’ but include annotation for label ‘B’. Thus, such sample is only excluded when computing AP for ‘A’ but will be included for ‘B’.

We construct unseen labels such that they have at least 75 annotated samples in test set for reliable evaluation.