customresources#AwsCustomResource: Lambda logs recreated after deletion

[metapox]

Describe the bug

I configured a LogGroup with a removalPolicy set to DESTROY and assigned it to an AwsCustomResource.

When I delete the stack, the LogGroup is deleted initially but is quickly recreated and persists.

It seems likely that the LogGroup is being recreated because asynchronous Lambda logs (from the AwsCustomResource) are being written after the LogGroup has been deleted.

Regression Issue

• [ ] Select this option if this issue appears to be a regression.

Last Known Working CDK Version

No response

Expected Behavior

The LogGroup should be deleted only after confirming that Lambda (AwsCustomResource) has finished writing logs.

Current Behavior

I set up an AwsCustomResource with a LogGroup that has the removalPolicy set to DESTROY. When the stack is deleted, the LogGroup appears to remain, even though the retention period is different and only the final Lambda execution logs are present.

This suggests that the LogGroup was deleted and then recreated due to the asynchronous writing of logs after the Lambda's execution.

Reproduction Steps

import * as cdk from 'aws-cdk-lib';
import { Construct } from 'constructs';
import { AwsCustomResource, AwsCustomResourcePolicy, PhysicalResourceId } from 'aws-cdk-lib/custom-resources';
import { aws_logs as logs } from 'aws-cdk-lib';
import * as sqs from 'aws-cdk-lib/aws-sqs';

const account = process.env.CDK_DEFAULT_ACCOUNT;
const region = process.env.CDK_DEFAULT_REGION;
const env = { account: account, region: region };

class StackPolicyApplierStack extends cdk.Stack {
    constructor(scope: Construct, id: string, props: cdk.StackProps, targetStack: cdk.Stack) {
        super(scope, id, props);
        const log = new logs.LogGroup(this, 'StackPolicyApplierLogGroup', {
            retention: logs.RetentionDays.ONE_DAY,
            removalPolicy: cdk.RemovalPolicy.DESTROY,
        });

        new AwsCustomResource(this, "StackPolicy-" + targetStack.stackName, {
            timeout: cdk.Duration.minutes(5),
            logGroup: log,

            onCreate: {
                service: 'CloudFormation',
                action: 'setStackPolicy',
                parameters: {
                    StackName: targetStack.stackName,
                    StackPolicyBody: JSON.stringify({
                        "Statement": [
                            {
                                "Effect" : "Allow",
                                "Action": "Update:*",
                                "Resource" : "*",
                                "Principal": "*",
                            },
                        ]
                    }),
                },
                physicalResourceId: PhysicalResourceId.of("StackPolicy-" + targetStack.stackName)
            },
            policy: AwsCustomResourcePolicy.fromSdkCalls({
                resources: [targetStack.stackId]
            })
        });
    }
}

class QueueStack extends cdk.Stack {
    constructor(scope: Construct, id: string, props?: cdk.StackProps) {
        super(scope, id, props);

        const queue = new sqs.Queue(this, 'QueueStack', {
            queueName: 'test-queue',
            visibilityTimeout: cdk.Duration.seconds(300)
        });
    }
}

const app = new cdk.App();

const queueStack = new QueueStack(app, 'QueueStack', { env: env });
const queueStackPolicyApplierStack = new StackPolicyApplierStack(app, 'QueueStackPolicyApplierStack', { env: env }, queueStack);

1. Deploy the stack
2. Delete the QueueStackPolicyApplierStack
3. The LogGroup (StackPolicyApplierLogGroup) remains.

Possible Solution

No response

Additional Information/Context

No response

CDK CLI Version

2.158.0

Framework Version

No response

Node.js Version

v20.17.0

OS

MacOS

Language

TypeScript

Language Version

No response

Other information

Even though I introduced a dependency between the LogGroup and the AwsCustomResource, the result remains unchanged.

[ashishdhingra]

@metapox Good afternoon. Unfortunately, I'm unable to reproduce the issue. Once CDK submits the delete stack request, CloudFormation takes over and handles the resource deletion. The CloudFormation would display the LogGroup has been deleted. Sometimes, the log streams take a while to be reflected in CloudFormation in the configured region, so it could be a timing issue.

Thanks, Ashish

[github-actions[bot]]

This issue has not received a response in a while. If you want to keep this issue open, please leave a comment below and auto-close will be canceled.