Closed akashAD98 closed 2 years ago
- this repo getting 70 fps on CPU https://github.com/neuralmagic/sparsezoo, just curious can we also do Pruning + Quantization to get more fps?
Yes. Pruning and quantization would help to increase FPS at inference time. Care must be taken to avoid accuracy drop, though. Check out NVIDIA TAO to see if it fits your needs.
- is there any script for converted onnx inference?
For the MODNet example in this repo, I made a test_onnx.py script which does inference with the ONNX model. You could reference it and develop your own code.
1.this repo getting 70 fps on CPU https://github.com/neuralmagic/sparsezoo, just curious can we also do Pruning + Quantization to get more fps? 2.is there any script for converted onnx inference?