search

khadas / ksnn

Khadas Software Neural Network
20 stars 10 forks source link

Performance degradation after memory leak issue is been resolved. #1

Open AatishLanghee opened 2 years ago

AatishLanghee commented 2 years ago

Hello,

Can you please check the following issues I am facing :

  1. We have tested Tiny Yolov3 model ( 416 * 416 ) trained on COCO dataset, there is a performance degradation we have observed in terms of FPS and GPU/NPU consumption.

Previously we were getting around 24 FPS and approx. 40% GPU/NPU consumption for Tiny Yolov3 model. Now, after memory leak issued is been resolved, we are getting around 17 FPS and approx. 10% GPU/NPU consumption for Tiny Yolov3 model.

The same is been observed with our custom trained models.

Is there any way we can improve the NPU/GPU consumption and can get higher FPS?

  1. Following parameters are not working while using conversion tool: a. --device type ( GPU/CPU ) b. --quantized-dtype ( perchannel_symmetric_affine / symmetric_affine / asymmetric_quantized)

  2. If we use INT8 for quantization, accuracy reduces. And for INT16, we are getting segmentation fault.

yan-wyb commented 2 years ago

I have solved this problem, you can try it with the latest SDK and KSNN