aws / aws-cdk

The AWS Cloud Development Kit is a framework for defining cloud infrastructure in code
https://aws.amazon.com/cdk
Apache License 2.0
11.68k stars 3.93k forks source link

(custom-resources): exceptions are not surfaced in cloudformation #31472

Closed ben-lee-zocdoc closed 1 month ago

ben-lee-zocdoc commented 1 month ago

Describe the bug

The doc says to Specifically, to report success or failure, have your Lambda Function exit in the right way: return data for success, or throw an exception for failure.

When our lambda throws an exception, the details are not surfaced in the Cloudformation dashboard. Instead, it says a generic Received response status [FAILED] from custom resource. Message returned: Error: Uncaught lambda exception,....

Regression Issue

Last Known Working CDK Version

No response

Expected Behavior

I expect the Reason to be populated with the Exception message, something like "Received response status FAILED from custom resource. Message returned: ." where the reason is lambda exception.

Current Behavior

We are seeing a generic error

Received response status [FAILED] from custom resource. Message returned: Error: Uncaught lambda exception, execution stopped Logs: /aws/lambda/my-lambda-function at invokeUserFunction (/var/task/framework.js:2:6) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async onEvent (/var/task/framework.js:1:369) at async Runtime.handler (/var/task/cfn-response.js:1:1676) (RequestId: abcdef-ghij-1234-5678-333f1c96d6d3)

Reproduction Steps

CDK code:

export class MyResource extends Construct {
    constructor(scope: Construct, id: string) {
        super(scope, id);

        const crProps: CustomResourceProps = {
            resourceType: 'Custom::resource',
            serviceToken: Fn.importValue(
                'LambdaFunctionArn'
            )
        };
        const resource = new CustomResource(this, 'custom', crProps);
    }
}

Our dotnet Lambda code:

public async Task<CustomResourceResponse<ResponseContract>> OnEvent(
    CustomResourceRequest<CustomResourceProperties> request,
    ILambdaContext context
)
{
    throw new Exception("I want this reason to show up");
}

Possible Solution

No response

Additional Information/Context

No response

CDK CLI Version

2.145.0

Framework Version

No response

Node.js Version

20

OS

Mac

Language

TypeScript

Language Version

4.8.3

Other information

The lambda is in net8.0

pahud commented 1 month ago

AWS CDK has a custom resource provider framework that comes as the Provider class as a wrapper for any user-provided lambda function and eventually exposed as a single provider to the native CFN custom resource. This means:

  1. From CFN's perspective, there would be a CustomResource with a provider lambda function associated.
  2. That lambda function is actually the provider wrapper that calls user function(onEvent handler).
  3. When onEvent handler returns to the provider, provider evaluate it and callback via cfn-response to CFN endpoint(a S3 endpoint URL).

The advantage is

  1. onEvent user handler just return a JSON object to the provider and provider takes anything rest.
  2. onEvent should handle exception when possible, however, if some unexpected exception is thrown, the provider framework should be able to capture that and gracefully callback cloudformation as resource fails to be created.
  3. you should be able to see all logs in cloudwatch logs as the provider framework would put necessary logs there.

However, based on your provided code:

export class MyResource extends Construct {
    constructor(scope: Construct, id: string) {
        super(scope, id);

        const crProps: CustomResourceProps = {
            resourceType: 'Custom::resource',
            serviceToken: Fn.importValue(
                'LambdaFunctionArn'
            )
        };
        const resource = new CustomResource(this, 'custom', crProps);
    }
}

Looks like you are not using CDK provider framework, instead, you are baking your own provider by passing serviceToken from an imported LambdaFunctionArn. We can't see how that lambda is created. Not sure if you use the provider framework or not.

If you actually didn't use provider framework, you will be responsible for:

  1. all exception handling in your own lambda
  2. you have to call back to cloudformation own your own, which provider framework takes care of you if you use the framework.

So please clarify if you are using the provider framework for cdk custom resources.

github-actions[bot] commented 1 month ago

This issue has not received a response in a while. If you want to keep this issue open, please leave a comment below and auto-close will be canceled.

ben-lee-zocdoc commented 1 month ago

We are not using the provider framework for the lambda. Thanks for clarifying.

github-actions[bot] commented 1 month ago

Comments on closed issues and PRs are hard for our team to see. If you need help, please open a new issue that references this one.