search

ceccocats / tkDNN

Deep neural network library and toolkit to do high performace inference on NVIDIA jetson platforms
GNU General Public License v2.0
718 stars 209 forks source link

tkDNN and YoloV4(AlexeyAB) are not the same #103

Closed harsco-jfernandez closed 3 years ago

harsco-jfernandez commented 4 years ago

Guys,

The wrong objects are being detected with tkDNN that are not with AlexeyAB repo. In my case, it seems that classifications were being swapped. So AlexeyAB repo works consistently at multiple thresholds, but tkDNN consistently fails at multiple thresholds. Thre is something wrong/different.

I am using YoloV4 tiny 3l(3 layer model) model and the best weights Darknet generated. I had the intuition that this was happening, but this time I did a proper comparison between the two libraries, and by proper, I mean several times to be absolutely certain.

Not sure if this is a bug or a limitation of TensorRT, but certainly, AlexeyAB library produces a better result at a lower framerate of course.

Hope you can find a fix to it and I realize this will be difficult without a smoking gun.

mive93 commented 4 years ago

Hi @harsco-jfernandez,

thank you for your hint. I already knew there was a little problem with the mAP, and I have spent the last 3 days trying to figure out where the problem was;.

Here's my answer : https://github.com/AlexeyAB/darknet/issues/5354#issuecomment-669739855. Moreover, here's the current mAP results of tkDNN https://github.com/ceccocats/tkDNN#map-results.

Now the outputs of tkDNN is almost the same :) (+- 0.0001)

harsco-jfernandez commented 4 years ago

Hello @mive93

Thank you so much!