can not get the satisfactory result on caltech pedestrian detection

[liuwei16]

I follow the default parameters to train the models for caltech dataset in ./mscnn-7s-720, the final caffemodel is about 233.0MB, while the author's is about 348.3MB in ./mscnn-7s-720-pretrained, where is the problem ? The detection result on my trained model is unsatisfactory.

[zhaoweicai]

For the model difference, have a look at https://github.com/zhaoweicai/mscnn/issues/23

For unsatisfactory results, what is performance you got? I just re-run the exact experiments 4 times today, and the miss rate is 10.33$\pm$0.19. It looks stable to me. If you get the miss rate >12, most likely your experiments have some problems.

[liuwei16]

Yes， thank you very much. Could you give me your experiment setting, like initial learning rate of two stages, the input size, with deconvolution layer, dimension conduction layer or no? Thank you very much. By the way, the bounding box de-normalization is not conducted when testing your provided model, but is required when testing our traind model, I wonder what the difference is.

[thhung]

@liuwei16 did you manage to get the same result 10.33 miss rate?

[thhung]

Hello @zhaoweicai , I am evaluating your pretrained model on the Caltech dataset for 1xtest? For different evaluation code, I only can get around 22% miss rate meanwhile, I expected it is ~10%. Could you provide more details about your evaluation? Thanks.

The only path of your matlab code I changed to the format of the output:

 fileID = fopen(['/path/to/output.txt'],'w');
 for i=1:nImg
     detected = cell2mat(final_detect_boxes(i,:));
     for j=1:size(detected,1)
         bbx = mat2str(ceil(detected(j,2:5)));
         fprintf(fileID,[char(image_list(i).name) ' ' bbx(2:end-1) ' ' num2str(detected(j,6)) ' \n']); 
     end
 end