serverless / examples

Serverless Examples – A collection of boilerplates and examples of serverless architectures built with the Serverless Framework on AWS Lambda, Microsoft Azure, Google Cloud Functions, and more.
https://www.serverless.com/examples/
Other
11.41k stars 4.46k forks source link

Lambda was unable to decrypt the environment variables because KMS access was denied #279

Open mohitkale opened 6 years ago

mohitkale commented 6 years ago

Dear Author,

For some strange reasons only the GET SINGLE TODO ITEM request is not working while all other APIs are working fine (i.e., LIST, CREATE, UPDATE, and DELETE).

I am getting this error, in the API Gateway console.

Reference Example: https://github.com/serverless/examples/tree/master/aws-node-rest-api-with-dynamodb

Endpoint response body before transformations: {"Message":"Lambda was unable to decrypt the environment variables because KMS access was denied. Please check the function's KMS key settings. KMS Exception: AccessDeniedExceptionKMS Message: The ciphertext refers to a customer master key that does not exist, does not exist in this region, or you are not allowed to access.","Type":null}

I am using same ITEM ID in both GET and DELETE methods, the DELETE method works but the GET method throws an Internal Server Error (stack trace as mentioned above).

Please suggest.

DenysVyskrebetsTR commented 2 years ago

manually changing lambda role to something else on the web portal and then back to the original role fixed the thing

JCapriotti commented 1 year ago

For anyone looking for a fix, there's a great write-up of the problem here: https://www.lastweekinaws.com/blog/the-sneaky-weakness-behind-aws-managed-kms-keys/

A brief summary is that this issue can occur if you delete and recreate the IAM role used by a Lambda function. The workarounds mentioned above seem to work: either update the lambda's role or recreate the lambda.

To avoid this altogether, one should avoid removing the IAM role used by a lambda (if possible) or use a customer managed key for encryption of the environment variables.

domengabrovsek commented 1 year ago

@liampauling thank you, this was a lifesaver!

https://github.com/serverless/examples/issues/279#issuecomment-420387109

miguellgramacho96 commented 10 months ago

Still happening to this day. Had to manually change the iam role to something else, saving and then changing back like @liampauling shared.

Fibio commented 7 months ago

I just had the same problem and as people mention here: it is related with redeployment using the same role name.

I did solved it by: IAM -> Roles -> $YourRoleNameHere -> Revoke Sessions -> Revoke active sessions

I hope it helps.

thank you a lot !!!

lagouyn commented 3 months ago

This helped me with my particular KMS/lambda issue, which occurred after my lambda role had gotten deleted, and I redeployed a replacement for that role: https://repost.aws/knowledge-center/lambda-kmsaccessdeniedexception-errors

felipegabry commented 3 months ago

I've deployed my lambdas with serverless framework and I got this only for one function, but not for the others. All functions are using the same role. Manually changing role in AWS for the function with this issue, to some other random role, and back to the original role fixed the problem. If it helps the one that was not working was triggered by Http GET, the one that worked was triggered by Http POST

Still working, thanks