Yolov8n model optimization

[RiccardoSegala04]

Hi, I recently retrained a yolov8n model following the instructions I found on the hailo model zoo GitHub repository. It works, but I'm trying to improve the performance of the model on hailo, since mine reaches approximately 220fps, and the model I could find on the repository (still yolov8n), reaches more than 700fps.

I tried optimizing my model using a compression level of 4 (just to see if the frame rate increases), but it tells me that my model has too little parameters to apply such compression. My model has less classes than the original one (mine has 5, and the original 80), so it has less parameters.

reading the documentation, I saw this way of forcing the compression level: model_optimization_config(compression_params, auto_4bit_weights_ratio=0.8), and it compiles, but when I try a benchmark of the model the performance do not increase, and the accuracy is a lot lower.

How can I increase the performance in fps of my model without loosing too much accuracy?

Thank you for your time

[omerwer]

Hi @RiccardoSegala04, Basically, when you apply 4-bit weights you gain more performance (e.g FPS) but you lose accuracy, as the weights are working with less data. To increase performance without loss of FPS, you can using one of our optimization algorithms (Finetune or Adaround) during the optimize step in the Dataflow Compiler pipeline. Please notice that each of those algorithms requires a GPU and at least 1024 images.

You can see how to use these algorithms in the Dataflow Compiler user guide located in the Developer Zone-->Documentation in the Hailo website.

Regards,

[GinRyan]

You can try to add --performance switch when you use hailomz to convert model.