Open starcrest opened 7 years ago
Have the exact same results here. Would also be interested in the cause of this.
It seems this might be because the readme reported values are actually evaluated on the train data. See #48
@iwcvanderlinden The default checkpoint can reach 82% on VOC07 test data. So, the result maybe is real. However, the given weight factors do not match the reported results.
I run the available weight in directory checkpoints named "ssd_300_vgg.ckpt". The result on VOC07 Test are AP_VOC07/mAP[0.82107575232572283] AP_VOC12/mAP[0.8562692165386977]
@huunhan312 I run the "ssd_300_vgg.ckpt" got the result on VOC07_test,mAP = 0.822. But after a month I rerun “ssd_300_vgg” on both voc07_test and VOC07_trian, got mAP = 0.736. I have no idear about this. Can you help me.
guess the ssd_300_vgg.ckpt
model is converted from caffe-ssd model... caffe-ssd
The evaluation results I obtained:
ssd_300_vgg.ckpt
AP_VOC07/mAP[0.82107502332759208]
AP_VOC12/mAP[0.8562692241613149]
VGG_VOC0712_SSD_300x300_iter_120000.ckpt
AP_VOC07/mAP[0.59928033396046509]
AP_VOC12/mAP[0.60921385591790489]
VGG_VOC0712_SSD_300x300_ft_iter_120000.ckpt
AP_VOC07/mAP[0.74313215436144464]
AP_VOC12/mAP[0.76659691986281664]
VGG_VOC0712_SSD_512x512_ft_iter_120000.ckpt
AP_VOC07/mAP[0.77656339500846039]
AP_VOC12/mAP[0.80274009098080834]
@huangh12 Could you help me? I'm confused that how to compute AP_VOC12/mAP. Actually, I only use VOC07 TEST to test? Thank you?
@TingZhang08 Hello, just run the eval_ssd_network.py as the readme.md said, you should get the mAP results once finished. On the principle behind mAP calculation, you can refer to this website for details.
I downloaded the models from the evaluation section and followed the instructions there to run
eval_ssd_network.py
.I get the following numbers: CHECKPOINT_PATH=./checkpoints/VGG_VOC0712_SSD_300x300_iter_120000.ckpt AP_VOC07/mAP[0.59928032818577548] AP_VOC12/mAP[0.60921401213750548]
CHECKPOINT_PATH=./checkpoints/VGG_VOC0712_SSD_300x300_ft_iter_120000.ckpt AP_VOC07/mAP[0.74313309022098] AP_VOC12/mAP[0.76659688181365315]
CHECKPOINT_PATH=./checkpoints/VGG_VOC0712_SSD_512x512_ft_iter_120000.ckpt AP_VOC07/mAP[0.77656339926263329] AP_VOC12/mAP[0.80273876556779]
So that's mAP of 0.599, 0.743, and 0.777. The readme reports mAP of 0.778, 0.817, and 0.837 respectively. Am I misunderstanding? Can you help me understand the discrepancy?
Thanks a lot for making this project available!