search

lucas-ventura / CoVR

Official PyTorch implementation of the paper "CoVR: Learning Composed Video Retrieval from Web Video Captions".
https://imagine.enpc.fr/~ventural/covr/
MIT License
80 stars 7 forks source link

The inquiry about test CIRR dataset #19

Closed gersys closed 3 months ago

gersys commented 3 months ago

Hi, thank you for this wonderful work.

I appreciate you providing the code.

I wonder how to calculate the recall performance of CIRR

I ran the code, but I think the code saves the image list of top-50 similarities, doesn't calculate the recall performance

So I checked the annotations of CIRR test-1, However, there is no label of the target image, only 'members'.

How do I calculate the recall performance of CIRR?

It would be beneficial to me

lucas-ventura commented 3 months ago

Hi,

There are no labels for the CIRR test set because they are not public. When you run python test.py test=cirr (or train.py with test=cirr), it will generate a recalls_cirr.json in the output folder. You will need to upload this to https://cirr.cecs.anu.edu.au/test_process/ to get the test results.

Let me know if you have any further questions.

Best regards, Lucas

gersys commented 3 months ago

Thank you for fast and kind reply :)