search

tusen-ai / simpledet

A Simple and Versatile Framework for Object Detection and Instance Recognition
Apache License 2.0
3.08k stars 488 forks source link

How to measure the inference time of the int8 quantified model? #283

Closed DuZzzs closed 4 years ago

DuZzzs commented 4 years ago

Thank you for your work. When I reproduce the int8 quantified example, how do I get the inference time of the int8 quantized model? I ran detection_test.py and did not observe speedups. Run a file? Does anyone know? Thanks.

This table: ( https://github.com/TuSimple/simpledet/tree/master/config/int8) dtype time(ms) minival mAP
fp32 260 35.7
int8 100 35.8

my environment: gpu : 2060 tensorrt 7.0

RogerChern commented 4 years ago

The inference time is measured on an internal platform based on trt. The INT8 quantization training will give you an INT8 model, but the inference software stack is beyond the current scope of SimpletDet.

On Wed, Dec 25, 2019 at 12:41 PM DuZzzs notifications@github.com wrote:

Thank you for your work. When I reproduce the int8 quantified example, how do I get the inference time of the int8 quantized model? I ran detection_test.py and did not observe speedups. Run a file? Does anyone know? Thanks.

This table: ( https://github.com/TuSimple/simpledet/tree/master/config/int8) dtype time(ms) minival mAP
fp32 260 35.7
int8 100 35.8

my environment: gpu : 2060 tensorrt 7.0

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/TuSimple/simpledet/issues/283?email_source=notifications&email_token=ABGODH4NIMYY74E66BSCGV3Q2LP75A5CNFSM4J7CKGGKYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4ICSD4IQ, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABGODHYR2QH5RTXY2WNMRRDQ2LP75ANCNFSM4J7CKGGA .

DuZzzs commented 4 years ago

I see. Thank you.