search

zhaoweicai / mscnn

Caffe implementation of our multi-scale object detection framework
404 stars 211 forks source link

Having trouble reproduce the evaluation result for the pretrained model #105

Closed liz2020 closed 4 years ago

liz2020 commented 4 years ago

@zhaoweicai When I use the run_cascademscnn.m file to get the inference for the new cascade mscnn pre-trained model provided, I got plenty of prediction results for cars even when there is only one car in the gt. Is this behavior expected? The attached file shows the inference for image 000001. Moreover, after I modify and run the writeDetForEval.m and evaluate_object, the evaluation result is close to only 0.06%. I am not sure what I am doing wrong, and I only change the lines related to the path.

The dataDir in my code might look weird because the dataset is stored in coco format, but I am sure it is pointing to the correct image directory and label directory.

Screen Shot 2019-09-19 at 1 14 00 AM Screen Shot 2019-09-19 at 1 42 00 AM Screen Shot 2019-09-19 at 1 37 08 AM Screen Shot 2019-09-19 at 1 22 34 AM

liz2020 commented 4 years ago

The line 87 of writeDetForEval.m assumes that the image with id equal to 'i' has its name put in the corresponding index of the train validation split file. This works when the image directory contains only the images related to the val.txt file and the names in val.txt file is sorted.

liz2020 commented 4 years ago

The first index in the file generated by run_cascademscnn.m is not the image name, but the index for the images exists in the JPEGImages folder. Thus, it has the same assumption as the writeDetForEval.m file that the folder only contains the images listed in the val.txt / train.txt file.