search

knorth55 / chainer-light-head-rcnn

[This project has moved to ChainerCV] Chainer Implementation of Light Head RCNN
MIT License
18 stars 1 forks source link

Reproduce original accuracy #1

Closed knorth55 closed 6 years ago

knorth55 commented 6 years ago

Scores for small objects is low. There should be some difference between original repo and this implementation.

Implementation mAP@0.5:0.95/all mAP@0.5/all mAP@0.75/all mAP:0.5:0.95/small mAP:0.5:0.95/medium mAP:0.5:0.95/large
Original 0.400 0.621 0.429 0.225 0.446 0.540
Ours 0.373 0.584 0.394 0.179 0.416 0.539
knorth55 commented 6 years ago

Image input size was small :( It is fixed in https://github.com/knorth55/chainer-light-head-rcnn/commit/996ffb725ea7a6296fefac750a4bdc0b83038562 and start training with new parameters.

knorth55 commented 6 years ago

I trained again, and it got much better. new version v1.0.1 is released with this trained model.

Implementation mAP@0.5:0.95/all mAP@0.5/all mAP@0.75/all mAP:0.5:0.95/small mAP:0.5:0.95/medium mAP:0.5:0.95/large
Original 0.400 0.621 0.429 0.225 0.446 0.540
Ours 0.391 0.607 0.419 0.212 0.428 0.541
Diff -0.9 -1.4 -1.0 -1.3 -1.8 0.1
aggpankaj2 commented 6 years ago

@knorth55 Hi, I able to run demo.py but while running train_multi.py i am getting segmentation fault (core dumped) . Can you please suggest what can be the possible reason behind this.

knorth55 commented 6 years ago

@aggpankaj2 Hi, I have no idea about your error. Can you open another issue and give me more detailed system output and information? And also please check if it is OpenMPI problem.

aggpankaj2 commented 6 years ago

@knorth55 OK i will give you more information about that in new issue. had you tried its original implementation given on this link (Original TensorFlow repository is zengarden/light_head_rcnn) If yes how much MAP you got on coco with tensorflow code i got only 5%.

knorth55 commented 6 years ago

I got almost same score in the paper with original tensorflow repo.

aggpankaj2 commented 6 years ago

Many thanks for your reply . I used their epoch_26.ckpt given by author. and simply test using python3 test.py -d 0 -se 26 but got 5% (tensorflow) given below. Can you suggest me where i am doing mistake

evaluation epoch 26 loading annotations into memory... Done (t=6.81s) creating index... index created! Loading and preparing results... DONE (t=3.93s) creating index... index created! Running per image evaluation... Evaluate annotation type bbox DONE (t=107.69s). Accumulating evaluation results... DONE (t=21.68s). Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.052 Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.080 Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.056 Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.033 Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.056 Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.070 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.040 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.060 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.063 Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.043 Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.067 Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.082

aggpankaj2 commented 6 years ago

@knorth55 can you give me some insight on above issue

knorth55 commented 6 years ago

I have no idea about your results... sorry about it. Can you check whether model parameter is correctly loaded?