search

philschmid / serverless-bert-huggingface-aws-lambda-docker

44 stars 16 forks source link

Works in local but not after deployed #2

Open MattiooFR opened 3 years ago

MattiooFR commented 3 years ago

Hello again,

My code works in local, I get my summary after 12s, but when I try after deployment, I get 504 Gateway Timeout. Probably because my Lambda function need more than 29s to run and send back the answer. Therefore I tried to use the async:true in the serverless.yml config file but it is still not working and when I look ad AWS logs I dont really understand as I'm not used to debug this.

Maybe you have some insight on how I can make it works ?

philschmid commented 3 years ago

The API-Gateway has a max timeout limit of 29seconds. If you rant into multiple 504, even after the function has loaded your model AWS Lamdba with an API Gateway is probably not suited.

XiaohanYa commented 2 years ago

Hi, thank you for the tutorial. I also encountered the same issue when using BertTokenizer and BertModel ("bert-base-uncased"). It took me 1.836s to run my functions locally, which I think should be fine for deployment. As it's my first time using aws, I wonder if you can share some insights on debugging. Many thanks.

philschmid commented 2 years ago

@XiaohanYa i recommend taking a look at the newest developments: https://www.philschmid.de/serverless-transformers-sagemaker-huggingface AWS added support for serverless Inference for Amazon SageMaker.