testing with DOTA dataset

[geobao]

Hi, thanks for your great work!

I split the testing with the default configuration split_configs/dota1_0/ss_test.json

Then I run the test as below:

EVAL_METRICS=mAP && \ CONFIG_FILE=configs/obb/oriented_rcnn/faster_rcnn_orpn_r50_fpn_1x_dota10.py && \ CHECKPOINT_FILE=ckpt/faster_rcnn_orpn_r50_fpn_1x_dota10_epoch12.pth && \ python tools/test.py ${CONFIG_FILE} ${CHECKPOINT_FILE} --eval ${EVAL_METRICS}

Everything runs fine. However I see that all AP scores are 0 because it does not pick up ground truth.

`root@673e18a920bd:/work/OBBDetection# EVAL_METRICS=mAP && CONFIG_FILE=configs/obb/oriented_rcnn/faster_rcnn_orpn_r50_fpn_1x_dota10.py && CHECKPOINT_FILE=ckpt/faster_rcnn_orpn_r50_fpn_1x_dota10_epoch12.pth && python tools/test.py ${CONFIG_FILE} ${CHECKPOINT_FILE} --eval ${EVAL_METRICS} load checkpoint from local path: ckpt/faster_rcnn_orpn_r50_fpn_1x_dota10_epoch12.pth [>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] 10833/10833, 4.2 task/s, elapsed: 2557s, ETA: 0s Merging patch bboxes into full image!!! Multiple processing [>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] 937/937, 39.0 task/s, elapsed: 24s, ETA: 0s Used time: 26.0 s

Start calculate mAP!!! Result is Only for reference, final result is subject to DOTA_devkit

+--------------------+-----+-------+--------+--------+ | class | gts | dets | recall | ap | +--------------------+-----+-------+--------+--------+ | large-vehicle | 0 | 23163 | 0.0000 | 0.0000 | | swimming-pool | 0 | 2190 | 0.0000 | 0.0000 | | helicopter | 0 | 439 | 0.0000 | 0.0000 | | bridge | 0 | 5914 | 0.0000 | 0.0000 | | plane | 0 | 4905 | 0.0000 | 0.0000 | | ship | 0 | 21624 | 0.0000 | 0.0000 | | soccer-ball-field | 0 | 1239 | 0.0000 | 0.0000 | | basketball-court | 0 | 784 | 0.0000 | 0.0000 | | ground-track-field | 0 | 901 | 0.0000 | 0.0000 | | small-vehicle | 0 | 37442 | 0.0000 | 0.0000 | | baseball-diamond | 0 | 1287 | 0.0000 | 0.0000 | | tennis-court | 0 | 2005 | 0.0000 | 0.0000 | | roundabout | 0 | 1076 | 0.0000 | 0.0000 | | storage-tank | 0 | 5646 | 0.0000 | 0.0000 | | harbor | 0 | 6926 | 0.0000 | 0.0000 | +--------------------+-----+-------+--------+--------+ | mAP | | | | 0.0000 | +--------------------+-----+-------+--------+--------+ `

Do I have to reformat the groundtruth of dota dataset before splitting? The configuration file points to the test partition that has been splitted, so everything is running by default. Am I missing something?

Thanks again

[jbwang1997]

ss_test.json doesn't specify ann_dir, so the dataset doesn't have ground-truth information. You can use ss_val.json to split your test set.

[geobao]

Thanks!

[fhaji75]

@geobao Can you solve this problem? If we use ss_val.json , we need labelTxt folder for test images.

[Windy4096]

DOTA does not provide label for test dataset Only the label of train and val dataset can we access