Improve inference speed on sagemaker serverless while preserving accuracy

tnc-ca-geo / animl-ml

Machine Learning resources for camera trap data processing

Other

4 stars 1 forks source link

Improve inference speed on sagemaker serverless while preserving accuracy #106

Closed rbavery closed 1 year ago

rbavery commented 1 year ago

Goal: get to 2-3 seconds per image.

according to yolov5 benchmarks and our past tests, inference speeds are possible, but with potential requirements to compromise on image size: https://github.com/ultralytics/yolov5/pull/6613

PR: https://github.com/ultralytics/yolov5/pull/6613

nathanielrindlaub commented 1 year ago

We got to ~3.5 seconds per image with the ONNX-compiled model deployed on Sagemaker Serverless - so chalking that up to a win for now (see https://github.com/tnc-ca-geo/animl-ml/pull/105). Nice job @rbavery thanks for your persistence on this!!