search

marcoslucianops / DeepStream-Yolo

NVIDIA DeepStream SDK 7.0 / 6.4 / 6.3 / 6.2 / 6.1.1 / 6.1 / 6.0.1 / 6.0 / 5.1 implementation for YOLO models
MIT License
1.38k stars 343 forks source link

PTQ is too bad,QAT is future!So ,any idea for QAT support? #499

Open tms2003 opened 6 months ago

tms2003 commented 6 months ago

I tested multiple models of deepstream-yolo, and the effect of int8 can be said to be very poor. I also tested qat on the classification model and it worked perfectly. If I generate trt weights using a project like yolov8-qat, can I switch directly to deepstream-yolo?

levipereira commented 2 months ago

QAT truly brings significant innovation to Model Quantization. Here's an implementation of YOLOv9 with nearly zero precision loss and a substantial reduction in latency.

https://github.com/levipereira/yolov9-qat