COCO val $mmap$ - Githubissues

ucas-vg / TOV_mmdetection

Include mmdetection version of TinyBenchmark. Official link.

https://github.com/ucas-vg/TinyBenchmark

Apache License 2.0

34 stars 2 forks source link

COCO val $mmap$ #9

Closed sxy1122 closed 2 years ago

sxy1122 commented 3 years ago

Sorry, I don't know which precision this COCO val $mmap$ represents. Which of AP0.5, AP0.75, AP0.5-0.95, APs, APm and APl is obtained by the network pre-training on the coco data set?

yinglang commented 3 years ago

It follows the common defination. mmAP is the mean iou of mean class of AP, so it means AP0.5-0.95.

sxy1122 commented 3 years ago

It follows the common defination. mmAP is the mean iou of mean class of AP, so it means AP0.5-0.95.

Thank you for your answer. I have another question. I ran it myself according to the code and instructions provided by you, but the result of pre training on coco was only 18.5, 1.1 less than 19.6 provided by you. Can you tell me why? Are there any changes to the specific settings?

sxy1122 commented 3 years ago

Also, does this refer to the range of various sizes mentioned in the paper? Then why is it different from what is said in the paper?

yinglang commented 3 years ago

swer. I have another question. I ran it myself according to the code and instructions provided by you, but the result of pre training on coco was only 18.5, 1.1 less than 19.6 provided by you. Can you tell me why? Are there any changes to the specific settings?

Do you add val dataset of coco to train? And if you use the trained weight as pretrian weight, the performance on TinyPerson is right. That's enough. Cause the performance on TinyCOCO is no meaning except for debug.

yinglang commented 3 years ago

Also, does this refer to the range of various sizes mentioned in the paper? Then why is it different from what is said in the paper?

The output of log is area range, not size range, where area=size*size.

sxy1122 commented 3 years ago

Does this code indicate that the verification set and the test set use the same data set? So why do you use the same data set for testing and verification? Won't this cause accuracy error?

And where can I find the files in this data? I downloaded the data set of coco2017, but I don't have this file, only the file you commented out.

yinglang commented 3 years ago

Yes, TinyPerson Version 1(this publish version) have no test set because of there is no enough data. We will fixed the problem for following version which may take some time to publish.

For instances_merge2017.json, it is just simply from merging all box annotation of coco. Maybe you can refer to this for some detail. (The given instances_simple_merge2014.json is same as instances_merge2017.json)