search

hybridgroup / gocv

Go package for computer vision using OpenCV 4 and beyond. Includes support for DNN, CUDA, and OpenCV Contrib.
https://gocv.io
Other
6.56k stars 862 forks source link

Using OpenVINO with AWS Lambda - Deploying OpenVINO as AWS Lambda Layer #517

Open StefanCepa opened 4 years ago

StefanCepa commented 4 years ago

Description

I would like to use OpenVINO as part of my AWS Lambda function in order to load and execute inference with models (packaged in .xml and .bin files) trained by OpenVINO framework.

Problem is, OpenVINO R1 that I am using takes ~900MB. My question is, what parts of OpenVINO do I need in order to load model and execute inference? I would like to package those parts and deploy them as AWS Lambda Layer.

Thanks in advance!

deadprogram commented 4 years ago

I have done something somewhat similar using MS Azure, but the code is not public, sadly.

However, I can say that an OpenVINO install is not known for small size. If you determine some tricks for shrinking please share!

deadprogram commented 4 years ago

Actually that info is public now: https://github.com/intel-iot-devkit/parking-lot-counter-go/blob/master/azure.md

Still does not address the size...

goto50ai commented 4 years ago

@StefanCepa have a look at ADLINK Edge, it has the apps for its edge platform that has connectivity for AWS, Firehose Lambda, OpenCV and Openvino, let me know if you would like to know more. https://www.adlinktech.com/en/Edge-IoT-Solutions-and-Technology It takes the complexity out of Edge computing and is very useful for Machine Vision

BorisChumichev commented 3 years ago

Now it's possible to mount EFS on Lambda. You can store your model distribution on EFS and access it from Lambda for inference: https://aws.amazon.com/blogs/compute/building-deep-learning-inference-with-aws-lambda-and-amazon-efs/