Issues with Converting YOLOv5 ONNX Model to HEF Format with and without NMS

[pcycccccc]

I have encountered some issues while modifying the official inference codes for (windows/yolov5) and (windows/yolov8) to support image inference. I tested the downloaded yolov5m_nms.hef and yolov5m_no_nms.hef files (both provided by Hailo), and I would greatly appreciate your assistance with the following concerns:

1. (windows/yolov8) Code: This code is capable of inferring the yolov8_nms.hef model, so I wondered if it could also infer the yolov5_nms.hef model. After replacing the model, I found that this code can indeed work with both models. However, I am not sure how to convert my trained yolov5m.onnx model into an HEF model with NMS. The official yolov5m.yaml configuration file seems to be for an older model, and even after modifying the nodes in yolov5m.yaml, I am still unsure if the provided yolov5m_nms.json is compatible with my model. During the conversion, I encountered the error The layer named conv84 doesn't exist in the HN. Could you kindly guide me on how to generate a yolov5m_nms.hef model with NMS? My model conversion command is: ‘hailomz compile --ckpt yolov5m.onnx --calib-path my_data/chunxin_image --model-script yolov5m.alls --classes 1 yolov5m’

2. (windows/yolov5) Code: I noticed that this example uses a yolov5.hef model without NMS, and this model type is Multi Context. To convert the yolov5m.onnx model for a custom dataset, I modified the yolov5m.alls and removed the content related to using the nms.json file. The modified yolov5m.alls content is as follows. However, when I performed inference with the code, the drawn bounding boxes were incorrect, as shown below: The bounding boxes for the generic model yolov5m_no_nms.hef were correct, as shown below.

   Could you kindly advise me on how to generate a yolov5m_no_nms.hef model without NMS?

Thank you very much for your assistance.

Best regards

[omerwer]

Hi @pcycccccc, To answer your questions:

1. when running hailomz compile, the MZ goes through the 3 Datafllow Compiler steps - parsing, optimization and compilation. In the optimization step, the MZ take the alls defined in the relevant YAML file or, as you did, a specific defined alls script. In the alls script, for the NMS on Hailo to be applied, you either define a path to the JSON config file or use a prebuilt option. For example, it can be: nms_postprocess(meta_arch=yolov5, engine=cpu) Or nms_postprocess('config_file_path', meta_arch=yolov5, engine=cpu)

Since your model in custom, you need to use the second option and update the relevant JSON, meaning you need to take the yolov5m_nms_config.json file in the MZ, and change it according to your model: You can have different number of classes, different anchors, different thresholds and different convolution layers that attached to the NMS ops. Only after you perform the correct changes, you will be able to compile a accurate HEF.

2. The yolov5 Windows example is pretty old and needs to be updated to support the NMS on Hailo. This is something that is in out tasks list. Regarding your question - it might be different anchors for your model that cause the bbox to shift. The multi context is not relevant to the accuracy of the model as it is just a way to have a large model, that exceeds the Hailo device's resources, run on the device. The way you created the HEF without NMS is the correct way to do it.

Regards,

[pcycccccc]

@omerwer Thank you for your response. I now understand why the model conversion with NMS was causing errors. The main reason was that several node names in the JSON file did not match my model. I rewrote the yolov5.alls file based on the yolov8.alls file, and during the model conversion process, I checked the hn file generated from the har file. I then modified several nodes in my nms.json. The model conversion was successful, and I obtained the correct inference results。