search

datvuthanh / HybridNets

HybridNets: End-to-End Perception Network
MIT License
594 stars 121 forks source link

inference latency #23

Closed ForestWang closed 2 years ago

ForestWang commented 2 years ago

the inference latency 37ms on V100 with FP16 from the paper. does it test with tensorrt or just python inference ?

and how about the speed with preprocess and NMS postprocessing?

thanks very much!

xoiga123 commented 2 years ago

  1. It's python inference. We are learning to port the model to tensorrt ourselves. You can check this superb piece of work for some quick benchmark in tensorrt. I believe he used a parser to parse our net, and since EfficientNet(Det) was fully parsed years ago, I doubt there's any performance drop compared to rebuilding HybridNets in tensorrt API.

  2. Unfortunately, we only benchmarked inference time. Nevertheless, just as other one-stage object detectors, pre+postprocessing should only take a fraction of a full pass.