search

tensorflow / models

Models and examples built with TensorFlow
Other
77.17k stars 45.76k forks source link

strange mAP for ssd_mobilenet_v1_coco #6794

Closed bulygin1985 closed 4 years ago

bulygin1985 commented 5 years ago

Model ssd_mobilenet_v1_coco gives mAP = 26.3. It is even larger then VVG + SSD (24.5)! I use COCO evaluator : https://github.com/cocodataset/cocoapi and COCO validation set. However, table mAP value is 21. If I tune the ssd_mobilenet_v1_coco on lr = 0.00001 (Adam optimizer) during 10 epoch then accuracy is decreased to mAP = 22.

Please, verify mAP accuracy of ssd_mobilenet_v1_coco. It is really larger than table value mAP = 21.

tensorflowbutler commented 5 years ago

Thank you for your post. We noticed you have not filled out the following field in the issue template. Could you update them if they are relevant in your case, or leave them as N/A? Thanks. What is the top-level directory of the model you are using Have I written custom code OS Platform and Distribution TensorFlow installed from TensorFlow version Bazel version CUDA/cuDNN version GPU model and memory Exact command to reproduce

tensorflowbutler commented 4 years ago

Hi There, We are checking to see if you still need help on this, as this seems to be an old issue. Please update this issue with the latest information, code snippet to reproduce your issue and error you are seeing. If we don't hear from you in the next 7 days, this issue will be closed automatically. If you don't need help on this issue any more, please consider closing this.

birdman9391 commented 4 years ago

Model ssd_mobilenet_v1_coco gives mAP = 26.3. It is even larger then VVG + SSD (24.5)! I use COCO evaluator : https://github.com/cocodataset/cocoapi and COCO validation set. However, table mAP value is 21. If I tune the ssd_mobilenet_v1_coco on lr = 0.00001 (Adam optimizer) during 10 epoch then accuracy is decreased to mAP = 22.

Please, verify mAP accuracy of ssd_mobilenet_v1_coco. It is really larger than table value mAP = 21.

mAP is evaluated from minival set. It means the model uses some validation data for training data. That may be the reason why the model have high accuracy.