search

MKLab-ITI / FIVR-200K

FIVR-200K dataset from the "FIVR: Fine-grained Incident Video Retrieval" [TMM 2019]
Apache License 2.0
78 stars 8 forks source link

Some videos are unavailable #2

Open UpCoder opened 5 years ago

UpCoder commented 5 years ago

Hi, I try to download videos by vids. But some video maybe unavailable. For example https://www.youtube.com/watch?t=4&v=KsJOvayT31o VID is d3xAXCQWkQo can you provide some way to download the failed videos?

Thanks

EthanGreen75 commented 4 years ago

meet the same problem, can we still use the incomplete video set to evaluate the map of the retrieval results? About 40000 videos download failure.

gkordo commented 4 years ago

Yes, you can still use the evaluation script. To do so, when you run the evaluation.py, you have to provide to the dataset_ids argument the path to file that contains the ids of the downloaded videos. For example:

python evaluation.py --result_file ./results.json --relevant_labels ND,DS --dataset_ids ./incomplete_dataset_ids.txt
Kennard123661 commented 4 years ago

@gkordo Hi, what happens if some of the queries are missing, will the evaluation script handle missing queries? Thank you!

gkordo commented 4 years ago

The evaluation script handles this case and ignores the missing queries. Hence, the evaluation metrics (mAP and PR-curve) are calculated based solely on the queries in the result file.