How to infer Test Results

[ghost]

Hello Loic, I successfully trained the model. I trained it for 50 epochs using a small number of batch size as well as small number of workers. I can visualize the data using Comet ML.

My question is about the testing: how should I infer the output of the test?

For example:

Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.021

 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.059

 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.010

 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.043

 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.021

 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.021

 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.034

 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.085

 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.116

 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.208

 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.085

 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.090

I currently have this output. I am specifically interested in the mAP score. Can you please share how to calculate the mAP score after training a model?

[loiccordone]

Hello wozcoder,

I'm not sure to understand your question. What we call "mAP score" corresponds to the first line of your output. It's 0.021. It still seems very low compared to the results I obtained with larger batch size though.

[ghost]

Yes, I am trying to figure out why. I will retry and increase the batch size this time. My latest result with 8 epochs w 16 batch size is 9% mAP.

Can you share the exact command you passed to the command line in order to obtain 96% mAP score with the vgg11?

I am trying to obtain the same results as you. If possible would you be able to share pretrained models with me? wozkoz@protonmail.com

[loiccordone]

I think you misunderstood the results I obtained. I obtained 96% classification accuracy on the Prophesee GEN1 classification task, not the object detection task. I obtained 0.174 mAP score with VGG-11 on the object detection task.

The exact commands I used are already shared in the README of this repository. The pretrained models will be available soon to everyone.

[ghost]

Hmm. I see, which model achieved the highest object detection accuracy? Thanks