Invoking Inactive lambda does't raise ResourceNotReadyException

[luzzodev]

Describe the bug

According to the documentation, invoking synchronously lambda that has Inactive state should throw Lambda.Client.exceptions.ResourceNotReadyException, instead ClientError is raised.

By analyzing the code in boto library and the way on how errors are built, i founded that aws api response is {'Message': 'ERROR: Lambda is initializing your function. It will be ready to invoke shortly.', 'Code': 'CodeArtifactUserPendingException'}

CodeArtifactUserPendingException is not mapped to any Lambda.Client.exceptions and this will cause to have a default fallback on ClientError

Expected Behavior

Invoking lambda with a status of inactive or pending should throw Lambda.Client.exceptions.ResourceNotReadyException according to the documentation

Current Behavior

By invoking an inactive lambda, the api will return

{'Message': 'ERROR: Lambda is initializing your function. It will be ready to invoke shortly.', 'Code': 'CodeArtifactUserPendingException'}

Looking at from_code function in botocore.errorfactory.py

def from_code(self, error_code): return self._code_to_exception.get(error_code, self.ClientError)

error_code in input is CodeArtifactUserPendingException and it is not listed in self._code_to_exception so self.ClientError is returned.

Reproduction Steps

Just invoke lambda that is inactive

Possible Solution

No response

Additional Information/Context

No response

SDK version used

boto3==1.21.29, boto3==1.28.7

Environment details (OS name and version, etc.)

macOs 12.4, windows 10, ubuntu 22.04

[RyanFitzSimmonsAK]

HI @luzzodev, thanks for reaching out and for your patience. Are you still having this issue? I'm having trouble reproducing this behavior; could you mention how you're getting inactive Lambdas in your testing?

[luzzodev]

@RyanFitzSimmonsAK according to aws lambda documentation, <<"A function becomes inactive when it has been idle long enough for Lambda to reclaim the external resources that were configured for it. ">> . In my case, I have thousands of Lambdas configured with 10 GB of RAM and a 15-minute timeout, so it could be relatively easy for a Lambda to become inactive if it is not invoked for a while.

[RyanFitzSimmonsAK]

While I wait for some functions to become inactive so I can reproduce the issue, could you provide debug logs of the behavior? You can get debug logs by adding boto3.set_stream_logger('') to the top of your script, and redacting any sensitive information. Thanks!

[luzzodev]

At the moment, I can't provide logs because I don't have any inactive Lambdas. As soon as possible, I'll try to replicate the flow with the debug log.

[luzzodev]

@RyanFitzSimmonsAK , here is the requested log. I have tried using the latest version of boto, but the error still occurs. boto_lambda_log.log

From what is see this is what happen:

Invoking an inactive lambda will generate the following error code : "CodeArtifactUserPendingException" with a status of 409.

This error code in not reported in the configuration file located at:

botocore/data/lambda/2015-03-31/service-2

according to the api verson i'm using.

The given error code should be configured in:

• shapes key
• operations -> "Invoke" key

By the way, from what I can see, this issue might be caused by an undocumented error code returned by the API and this case should be handled as a LambdaNotReadyException.

[jrkarnes]

@RyanFitzSimmonsAK according to aws lambda documentation, <<"A function becomes inactive when it has been idle long enough for Lambda to reclaim the external resources that were configured for it. ">> . In my case, I have thousands of Lambdas configured with 10 GB of RAM and a 15-minute timeout, so it could be relatively easy for a Lambda to become inactive if it is not invoked for a while.

Given that a Lambda can become inactive if the timeout is reached without any invocations, if you don't need synchronous responses, it may be useful to invoke with a DLQ.

If you require a synchronous response, one of the things we've had luck with is building a /wake-up endpoint... but you may want to be careful with that because it puts your bill at the mercy of lambda timeouts tied to a public function endpoint. FWIW, we secure ours with an API gateway that has an allow-list of callers for the /wake-up endpoints on all our functions.