core: Unresolved stack-level tags cause errors when there are custom resources in the stack

Describe the bug

This issue is related to https://github.com/aws/aws-cdk/issues/29424 (CDK should emit an error when there are unresolved stack-level tags), but related to a bit more specific failure scenario.

The discussion in https://github.com/aws/aws-cdk/issues/29424 describes how it works to do things like Tags.of(myStack).add(Aws.STACK_NAME) to tag all the resources in a stack with some dynamic tag value (even if that value is unresolved at the top/stack-level). However, there are cases where the current functionality results in errors at deploy time, not just unresolved tokens in stack-level tags.

Expected Behavior

I expect (based on the discussion in https://github.com/aws/aws-cdk/pull/30022#issuecomment-2276754163) that I can tag a stack like Tags.of(myStack).add(Aws.STACK_NAME) and have all resources within the stack tagged, even unresolved tokens remain in the stack-level tags.

Current Behavior

If I tag a stack with some dynamic value, and that stack contains custom resources implemented with CustomResourceProvider (as does e.g. ec2.Vpc), then tagging all resources in a stack as described above results in an error. For example, in this case:

#!/usr/bin/env node
import 'source-map-support/register';
import * as cdk from 'aws-cdk-lib';
import * as ec2 from 'aws-cdk-lib/aws-ec2'
import { Construct } from 'constructs';

const app = new cdk.App();

class TagTestStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    const myVpc = new ec2.Vpc(this, 'Vpc');
    cdk.Tags.of(this).add('StackName', cdk.Aws.STACK_NAME)
  }

}
new TagTestStack(app, 'TagTestStack');

An error is returned for the IAM Role resources created by the custom resource provider because its tag contains invalid characters.

Reproduction Steps

See "Current Behavior" for a minimal reproducible example.

Possible Solution

I believe the reason this happens is because CustomResourceProvider creates e.g. its IAM Role "manually" with CfnResource like this. I believe this means the created resource does not have a TagManager and is not taggable by the aspect-oriented tagging machinery. Because it does not receive a tag for a given tag key from the CDK, that means the default CloudFormation behavior of propagating stack-level tags to all resources gets a chance to set the tag value, and it uses the unresolved token. This makes the error particularly confusing, because the error (the "root cause") of the CloudFormation deploy failure is associated with e.g. the IAM Role resource, even though there are no tags applied to it in the synthesized template.

Additional Information/Context

It seems intentional that CustomResourceProvider resources (like IAM Role and Lambda fn) are left out of the CDK's normal Tagging machinery, but if not, that could be a bug.

In any case, a couple workarounds today to tag all resources in a Stack but skip the top-level stack are:

myStack.node.children.forEach(construct => {
  cdk.Tags.of(construct).add(someTagKey, someDynamicValue);
});

cdk.Tags.of(myStack).add(someTagKey, someDynamicValue,
    {excludeResourceTypes: ['aws:cdk:stack']}
);

CDK CLI Version

2.147.2

Framework Version

No response

Node.js Version

18.15.0

OS

macOS

Language

TypeScript

Language Version

No response

Other information

No response

aws / aws-cdk