search

dingjiansw101 / AerialDetection

Apache License 2.0
796 stars 184 forks source link

DOAI1_5 performance is too low for the baseline model #3

Open tbuikr opened 4 years ago

tbuikr commented 4 years ago

Thanks for sharing the code!

I have tested the repo on DOAI 1.5 and submit it to the online evaluation task 1. But the result is too low. 

mAP: 0.07475452980149473
ap of each class: plane:0.010195412064570943, baseball-diamond:0.06983930026884663, bridge:0.01515151515151515, ground-track-field:0.03823953823953824, small-vehicle:0.042740171646495316, large-vehicle:0.12425185385654987, ship:0.03718538569783553, tennis-court:0.30461315579188125, basketball-court:0.07774816494045518, storage-tank:0.09090909090909091, soccer-ball-field:0.10014430014430015, roundabout:0.09976556394894504, harbor:0.0556564329231872, swimming-pool:0.03222999383810684, helicopter:0.09090909090909091, container-crane:0.006493506493506493

This is my script to reproduce the result above

python ./tools/test.py ./configs/DOTA1_5/faster_rcnn_RoITrans_r50_fpn_1x_dota1_5.py work_dirs/faster_rcnn_RoITrans_r50_fpn_1x_dota1_5/epoch_12.pth --out work_dirs/faster_rcnn_RoITrans_r50_fpn_1x_dota1_5/results.pkl
python ./tools/parse_results.py --config ./configs/DOTA1_5/faster_rcnn_RoITrans_r50_fpn_1x_dota1_5.py --type OBB

How could I fix it? Thanks

Note that, the inference for the demo image is still good.

P0009_out1

dingjiansw101 commented 4 years ago

Have you set the correct test json and images?

tbuikr commented 4 years ago

I used script to split train-val and test https://github.com/dingjiansw101/AerialDetection/blob/master/DOTA_devkit/prepare_dota1_5.py

dingjiansw101 commented 4 years ago

You could visualize the results on test set chips to see if anything wrong.

j93hahn commented 3 months ago

@dingjiansw101 @tbuikr I have the same problem. The paper claims 65.03 OBB mAP on DOTAv1.5, but I just submitted the pretrained model's results to the server and got 33 OBB.

What's the discrepancy?