YoloV5 C# detection is less accurate

[afisk]

I am finding that the object detection is different and maybe less accurate in C# with the onnx model than it was in Python with the PyTorch model. Is there any fix or reason for this? I have resized my images to be 640x640 to hopefully avoid any differences in image resizing. I also have the same confidence and IOU thresholds in both and get different results.

C#: Python:

C#: Python:

Any idea why the Python & PyTorch model seems to preform better?

[BobLd]

I can't really answer on the PyTorch/onnx difference as I'm no specialist, but I would expect the models to behave similarly... (also depending on how you did the conversion)

Below are stuffs I would invertigate:

• Is the C#/onnx model working for some pictures (in what you've displayed, it's always failing)? If it works sometime, could you post the differences in results?
• Are the raw outputs from the models identical (i.e. before post-processing of the outputs)? You will need to understand the Python code and print in some way the outputs before post-processing (here, I assume the outputs have the same shapes for both models...)
• Is post-processing done the same way? An important part of it is done here: https://github.com/BobLd/YOLOv4MLNet/blob/0104728917e3001e5b04feb794c3ffcb6dd8a819/YOLOv4MLNet/DataStructures/YoloV4Prediction.cs#L74-L108 The NMS stands for "non max suppression" and might be done differently in PyTorch...
• I'd try to display on pictures all the results from the C#/onnx, and try to undestand if the expected candidate bounding box are here.

Hope this helps

EDIT: I've just noticed that's you've edited your post and previously display some positive resutlts from the C#/onnx model and the scores were already differents. This makes me think that there is a difference in the post-processing step. Have a look at how the NMS is done in the Python code and maybe play with the scoreThres and iouThres parameters in the GetResults() function:

public IReadOnlyList<YoloV5Result> GetResults(string[] categories, float scoreThres = 0.5f, float iouThres = 0.5f)

[afisk]

I turned off the NMS (never let it delete a box) and it appears that it is still not finding the images.. do you think the .pt to onnx converter might be bad? I used the one that came with the ultralytics/yolov5 .

C# NMS at .45 - Finds right side deer C# no NML - finds right side deer Python .45 NMS - finds both deer

[BobLd]

okay that's strange... The scores are still different for the correctly found bounding box (0.88 in C# vs. 0.94 in Python)

• Can you also deactivate the IoU thresold if not already done and check the output? (comment the line below) https://github.com/BobLd/YOLOv4MLNet/blob/0104728917e3001e5b04feb794c3ffcb6dd8a819/YOLOv4MLNet/DataStructures/YoloV4Prediction.cs#L50
• Really don't know but I'd expect the converter to work fine.
• What about the pre-processing of the image? How do you feed the image to the onnx model? And how is it done in Python? Is there any colour balancing in the Python version...?
• Something else to try if possible would be to use the onnx model direcly from Python... Is ultralytics providing the Python code for the onnx model processing?

[afisk]

This is pretty much every image with the threshold turned off:

The pre-processing looks okay to me. There should be no resizing since I am feeding in the size of the model's inputs, 640x640. There is scaling: img/255. Python is nice and does most of the hard work and thinking for you :)

I'm not married to YoloV5 so maybe it would be easier to switch to YoloV4. With your YoloV4, how were you creating your onnx files and were all of your objects being found both in the original YoloV4 model and the Onnx?

[BobLd]

haha quite messy

• I'd be you I would try to understand what's going on with your current c# code, you seem to be very close. Try uncommenting your if (objConf <= scoreThres) continue; on put a low scoreThres, to someting like 0.1 and increasing it. You will see less and less bounding boxes and I'd hope to see the second deer... This should help understand...
• On my side I didn't trained my models, I was just reusing already existing onnx models. So I won't really be able to help here.
• As far as I remember, I did train a YoloV3 model and converted it to onnx. Maybe have a look here: https://github.com/BobLd/YOLOv3MLNet I have put the Python code I used for the conversion.

[afisk]

Darn. It starts detecting the second deer once the confidence score gets below 0.0003:

I'll go look through that YoloV3 and see if I can find something useful in that too! :)

[afisk]

YOU WERE RIGHT! Very close!! :) :) :) My pytorch model output is 10647x17classes (for 416x416 images ((52 x 52) + (26 x 26) + 13 x 13)) x 3 = 10647).. the converted model (onnx) has an output size of 25200x17 classes. I converted the pt to onnx converter to take in 416x416 images. The drawings are a little off, but the confidences are almost on point and the boxes show up in the relative areas of where the objects are in relation to one another. I'll try and get the bounding boxes to show in the correct places tomorrow and update here. Thank you very very much for all of your help (and encouragement) so far!!

C# 416x416: Python Original Model:

C# 416x416: Python Original Model:

[BobLd]

That's great news! This now seems to be a scaling issue. Just checking but make sure you've updated the following to match your dimensions: https://github.com/BobLd/YOLOv4MLNet/blob/0104728917e3001e5b04feb794c3ffcb6dd8a819/YOLOv4MLNet/DataStructures/YoloV4Prediction.cs#L31-L32

[afisk]

That was it! Works great now!! Thank you sooo much @BobLd !

[BobLd]

Great!