search

aws-samples / aws-serverless-for-machine-learning-inference

This is a sample solution for bringing your own ML models and inference code, and running them at scale using AWS serverless services.
MIT No Attribution
34 stars 9 forks source link

Editing lambda handler #7

Open InquestGeronimo opened 1 year ago

InquestGeronimo commented 1 year ago

Hi I'm trying to customize the lambda handler for real-time inference. However, If I alter the requirements.txt, deployment in cloudformation fails. For example, if I add boto3>=1.15.0 to the requirements.txt file deployment fails even though the local image boto3 installs just fine. Is the realtime inference stack sensitive to dependency changes?

Here is my requirements.txt (added boto3 at the end):

mxnet==1.6.0 numpy==1.19.5 urllib3==1.26.5 certifi==2022.12.7 idna==2.10 graphviz==0.8.4 chardet==3.0.4 boto3>=1.15.0

Is it possible for you to reproduce by editing the requirements.txt to see if it fails on your end as well? I'm using cdk version 2.59.0 (build b24095d) and using the ./install.sh flow.

FYI, if I try to deploy the default code/requirements it works fine. But if I alter the requirements file, deployment fails.

Thank you

InquestGeronimo commented 1 year ago

@medarg just following up on this thank you

medarg commented 1 year ago

@medarg just following up on this thank you

@InquestGeronimo What is the error thrown by CloudFormation?

At high level, if you're simply modifying the Lambda code and nothing else, you'll likely run into an error as that function is configured with a layer for the specified libraries and versions.

If you want to use any other code, you can either create another layer with ResNet and all of its dependencies, and ensure your Lambda deployment package (code + dependent libraries) are within Lambda's 50 MB limit (zipped) - 250 MB (unzipped).
if your code deployment package size is more, you have an option to upload it to S3 and download it while triggering the function invocation or put the function in Lambda compatible container, which allows up 10GB

InquestGeronimo commented 1 year ago

The error occurs on step 14 of 16. It says clouldformation failed to create lambda layer for realtime inference. The error i'm getting is strictly from only modifying the requirements.txt file not the lambda handler portion.

Is it possible for you to reproduce by adding boto3 like I did above?

medarg commented 1 year ago

@InquestGeronimo It's difficult to say exactly why the lambda layer creation failed without the exact error. One more thing you can try is to inspect the CloudTrail logs as soon as cloudformation stack fails for possible additional clues, as the root cause may vary.