search

CAPTAIN-WHU / DOTA_devkit

644 stars 185 forks source link

how to deal with 'difficulty' label #19

Closed bryan-bisonai closed 5 years ago

bryan-bisonai commented 5 years ago

I wonder how the evaluation script for test set deal with the "difficulty" labels. As written in the DOTA_devkit/dota-v1.5_evaluation_task2.py script code when difficulty is labeled as 1, which means the instance is difficult to be detected due to blur or too small, the instance is discarded although no penalty is incurred for detecting them.

Link to the code: https://github.com/CAPTAIN-WHU/DOTA_devkit/blob/4db7aeda8dea3d55fe4f5e839044e4b37e22b62d/dota-v1.5_evaluation_task2.py#L143

I would like to ask if this rule will be applied on the CVPR 2019, Detecting Objects in Aerial Images, competition as well.

dingjiansw101 commented 5 years ago

CVPR 2019 is different from DOTA-v1.0. Please see the notes in https://captain-whu.github.io/DOAI2019/evaluation.html.

bryan-bisonai commented 5 years ago

So according to the statement in the link you provided, it says Note, in the evaluation, the instances with a difficult label is also included, which is different from the DOTA-v1.0.

So... what is the main difference of a difficult label between DOTA-v1.0 and DOTA-v1.5?

dingjiansw101 commented 5 years ago

In the annotation of DOTA-v1.5, many small instances(even below 10 pixels) are also labeled.