(aws-servicecatalog): Wrong s3 file name when using asset in productstack with lambda based custom resource

[wzx0z]

Describe the bug

File path in asset.json is "asset.foo.bar" and object key is "foo.zip". However, in generated product template, S3Key becomes "bar.zip", which causes cloudformation to fail to find s3 object when provisioning a product.

asset.json: "288623806661fcbe44c12aa4b616582a3ded66381a5ba9d2e534355c69d4af49": { "source": { "path": "asset.288623806661fcbe44c12aa4b616582a3ded66381a5ba9d2e534355c69d4af49.241bc9c3c840f2b3310817506e9382190b3c31b5ecd74510d1e3f3f948e9febb", "packaging": "zip" }, "destinations": { "xxx": { "bucketName": "cdk-hnb659fds-assets-xxx", "objectKey": "288623806661fcbe44c12aa4b616582a3ded66381a5ba9d2e534355c69d4af49.zip", } } },

product.template.json: "LambdaFunction644AFC3F": { "Type": "AWS::Lambda::Function", "Properties": { "Code": { "S3Bucket": "asset_bucket_name", "S3Key": "241bc9c3c840f2b3310817506e9382190b3c31b5ecd74510d1e3f3f948e9febb.zip" }, }

Expected Behavior

product.template.json: "S3Key": "288623806661fcbe44c12aa4b616582a3ded66381a5ba9d2e534355c69d4af49.zip"

Current Behavior

"S3Key": "241bc9c3c840f2b3310817506e9382190b3c31b5ecd74510d1e3f3f948e9febb.zip"

Reproduction Steps

create lambda based custom resource in product stack.

Possible Solution

No response

Additional Information/Context

No response

CDK CLI Version

2.67.0 (build b6f7f39)

Framework Version

No response

Node.js Version

v16.17.0

OS

MacOS

Language

Python

Language Version

Python (3.9.15)

Other information

No response

[pahud]

Thank you for the report. Can you provide full working sample code that we can reproduce in our account?

[wzx0z]

@pahud Thank you for your reply. This issue might be related to "BundlingOptions", we use local bundling because we are running cdk in container. Below is my sample code.

stack.py

from typing import Any
from aws_cdk import (
    App, Stack, Environment,
    Duration, CustomResource,
    ILocalBundling, BundlingOptions, DockerImage

)
from constructs import Construct
from aws_cdk import RemovalPolicy
import aws_cdk.aws_servicecatalog as sc
import aws_cdk.aws_iam as iam
import aws_cdk.aws_s3 as s3
import aws_cdk.aws_lambda as lambda_
import shutil
import pathlib
import jsii

@jsii.implements(ILocalBundling)
class PythonLocalBundle:
    def __init__(self, soure):
        self.source = soure

    def try_bundle(self, output_dir, options):
        can_run_locally = True  # replace with actual logic
        if can_run_locally:
            # perform local bundling here
            print("Build lambda locally")
            print(f"Copy {self.source} to {output_dir}")
            shutil.copytree(self.source, output_dir, dirs_exist_ok=True)
            return True
        return False

class DebugProduct(sc.ProductStack):
    def __init__(self, scope, id, asset_bucket):
        super().__init__(scope, id,asset_bucket=asset_bucket)

        # create lambda role
        lambda_role = iam.Role(
            self,
            "LambdaRole",
            assumed_by=iam.ServicePrincipal("lambda.amazonaws.com"),
            managed_policies=[
                iam.ManagedPolicy.from_aws_managed_policy_name(
                    "service-role/AWSLambdaBasicExecutionRole"
                )
            ]
        )

        code_uri = pathlib.Path(__file__).parent.joinpath("runtime").resolve()

        # create lambda function
        lambda_function = lambda_.Function(
            self,
            "LambdaFunction",
            runtime=lambda_.Runtime.PYTHON_3_9,
            timeout=Duration.minutes(5),
            code=lambda_.Code.from_asset(
                str(code_uri),
                bundling=BundlingOptions(
                    local=PythonLocalBundle(soure=code_uri),
                    # Docker bundling fallback
                    image=DockerImage.from_registry("alpine"),
                    entrypoint=["/bin/sh", "-c"],
                    command=["bundle"],
                ),
            ),
            handler="lambda_function.lambda_handler",
            role=lambda_role
        )

        # create full control custom resource
        self.resource = CustomResource(
            self,
            "CustomResource",
            service_token=lambda_function.function_arn,
            removal_policy=RemovalPolicy.DESTROY,
            resource_type=f"Custom::CDKDebug",
            properties={},
        )

class DebugStack(Stack):
    def __init__(
        self,
        scope: Construct,
        id_: str,
        **kwargs: Any,
    ):
        super().__init__(scope, id_, **kwargs)

        asset_bucket_name = f"debug-asset-{self.account}-{self.region}"

        # create artifact s3 bucket
        asset_bucket = s3.Bucket(
            self,
            "DebugBucket",
            bucket_name=asset_bucket_name
        )

        # create service catalog protfolio
        portfolio = sc.Portfolio(
            self,
            "DebugPortfolio",
            display_name="Debug Portfolio",
            description="Potfolio for cdk debug",
            provider_name="cdk",
            message_language=sc.MessageLanguage.EN,
        )

        # create service catalog production
        product_from_stack = sc.CloudFormationProduct(
            self,
            "DebugProduct",
            product_name="DebugProduct",
            owner="cdk",
            description="Product for cdk debug",
            distributor="cdk",
            product_versions=[
                sc.CloudFormationProductVersion(
                    product_version_name="v1",
                    cloud_formation_template=sc.CloudFormationTemplate.from_product_stack(
                        DebugProduct(
                            self,
                            "DebugProductTemplate",
                            asset_bucket=asset_bucket,
                        ),
                    ),
                )
            ],
        )
        portfolio.add_product(product_from_stack)

app = App()
DebugStack(
    app,
    "DebugStack",
    env=Environment(
        account="CHANGE_TO_ACCOUNT_ID",
        region="CHANGE_TO_REGION",
    ),
)
app.synth()

runtime/lambda_function.py

def lambda_handler(event, context):
    return "CDK"

[wzx0z]

After a few tries, I found out that it works when I change "asset_hash_type" to "AssetHashType.OUTPUT", don't know why the default AssetHashType doesn't work.

lambda_function = lambda_.Function(
    ...
    code=lambda_.Code.from_asset(
        ...
        asset_hash_type=AssetHashType.OUTPUT
    ),
    ...
)

[pahud]

I am not sure if there's any limitation in your containerized environment but I am happy BundlingOptions works for you. Resolving this issue now. Feel free to open a new one if you have any other concern.

[github-actions[bot]]

Comments on closed issues and PRs are hard for our team to see. If you need help, please open a new issue that references this one.