Speed up ineference - Githubissues

HCA97 / Mosquito-Classifiction

7th place solution of Aicrowd Mosquito Alert Competition

GNU General Public License v3.0

1 stars 1 forks source link

Speed up ineference #5

Closed HCA97 closed 1 year ago

HCA97 commented 1 year ago

https://neuralmagic.com/blog/yolov8-detection-10x-faster-with-deepsparse-500-fps-on-a-cpu/
https://docs.ultralytics.com/modes/export/
https://docs.neuralmagic.com/get-started/transfer-a-sparsified-model/cv-object-detection
https://sparsezoo.neuralmagic.com/models/yolov8-m-voc_coco-pruned75?hardware=deepsparse-c6i.12xlarge&tab=3

HCA97 commented 1 year ago

PyTorch 2.0.1 and Python 3.10 speed up the inference a lot :P (We can run YoloV8-s (640x640) + ViT-L-14 (224x224) in around 1000ms) in old PyTorch version 1.13 I couldn't run YoloV8-s (1024x1024).

ONNX didn't helped
OpenVIVO f16 didn't help
TFlite f16 didn't help
Channel Last didn't help

All the libraries claimed they were faster but in the end, PyTorch was the fastest :P