search

ceccocats / tkDNN

Deep neural network library and toolkit to do high performace inference on NVIDIA jetson platforms
GNU General Public License v2.0
717 stars 209 forks source link

AP results confusing #192

Closed ionicaML closed 3 years ago

ionicaML commented 3 years ago

Hello, I have a question. I have seen on the YOLOv4 repo that the AP50 is 64.9%. However, in the MAP results section it shows that yolov4-tkdnn (416x416) has 0.695, or 69.5% AP50. How is this calculated? I feel like my presumption is wrong, but I can't figure out why. From what I understood, yolov4-tkdnn was only a speedup improvement, so results at AP50 should be around equal, maybe with a 1-2% lower AP.

mive93 commented 3 years ago

Hi @ionicaML indeed what you said it's right, tkDNN only optimize yolov4 in terms of execution time, the network and the weights are exactly the same.

In the table that you can find in our README you have the comparison of both the original yolo (darknet column) and our (tkDNN column), both computed via CodaLab (let's say official way) and tkDNN (our implementation of mAP computation).

As you can see original yolov4 AP50 is 0.705 (70,5%), while tkDNN is 0.710 (71,0%), which are almost the same. The reason why it is not the same as the one written on the repo/paper is just because probably they were using a greater confidence threshold (e.g. 0.05), while those tests have been performed with a conf threshold of 0.001.

I hope it's clearer now.

ionicaML commented 3 years ago

EDIT: I misunderstood your answer. It's all very clear now, thanks for your answer!