recall@5 COCO image-to-text (table 5)

[FiorenzoParascandolo1]

I'm trying to calculate the recall@5 on COCO image-to-text (table 5). I have some questions since I'm not getting the same results:

• Are the results calculated on the Karpathy split test?
• Since COCO has 5 captions per image, the recall@5 is computed considering a "true positive" if one of the five caption is in the top 5?

I tried using the Karpathy split test. I computed the recall@5 considering the recall@5 described above but I get the following results:

CLIP (openai): 77.2 vs 75 in the paper CE-CLIP: 75.5 vs 74.3 in the paper

[lezhang7]

Hi,

For benchmark coco retreival, we directly use https://github.com/LAION-AI/CLIP_benchmark/tree/main toolkit.