Open UpCoder opened 5 years ago
meet the same problem, can we still use the incomplete video set to evaluate the map of the retrieval results? About 40000 videos download failure.
Yes, you can still use the evaluation script. To do so, when you run the evaluation.py
, you have to provide to the dataset_ids
argument the path to file that contains the ids of the downloaded videos. For example:
python evaluation.py --result_file ./results.json --relevant_labels ND,DS --dataset_ids ./incomplete_dataset_ids.txt
@gkordo Hi, what happens if some of the queries are missing, will the evaluation script handle missing queries? Thank you!
The evaluation script handles this case and ignores the missing queries. Hence, the evaluation metrics (mAP and PR-curve) are calculated based solely on the queries in the result file.
Hi, I try to download videos by vids. But some video maybe unavailable. For example https://www.youtube.com/watch?t=4&v=KsJOvayT31o VID is d3xAXCQWkQo can you provide some way to download the failed videos?
Thanks