search

jkjung-avt / tensorrt_demos

TensorRT MODNet, YOLOv4, YOLOv3, SSD, MTCNN, and GoogLeNet
https://jkjung-avt.github.io/
MIT License
1.74k stars 545 forks source link

is there any way to get high fps on cpu ? #556

Closed akashAD98 closed 2 years ago

akashAD98 commented 2 years ago

1.this repo getting 70 fps on CPU https://github.com/neuralmagic/sparsezoo, just curious can we also do Pruning + Quantization to get more fps? 2.is there any script for converted onnx inference?

jkjung-avt commented 2 years ago
  1. this repo getting 70 fps on CPU https://github.com/neuralmagic/sparsezoo, just curious can we also do Pruning + Quantization to get more fps?

Yes. Pruning and quantization would help to increase FPS at inference time. Care must be taken to avoid accuracy drop, though. Check out NVIDIA TAO to see if it fits your needs.

  1. is there any script for converted onnx inference?

For the MODNet example in this repo, I made a test_onnx.py script which does inference with the ONNX model. You could reference it and develop your own code.