(CLI): Building & Publishing of assets happen even when there are no changes to them (aws-lambda-nodejs)

[piotrmoszkowicz]

Describe the bug

I've added custom mechanism to ensure we do not have to esbuild assets each time when there are no code changes. But on CLI level these assets are build and published anyways. I don't see any possibility to change that.

Building happens here: https://github.com/aws/aws-cdk/blob/34c7b9e7d882cd76a2b2f513781f9202385f4d30/packages/aws-cdk/lib/cdk-toolkit.ts#L175 Publishing: https://github.com/aws/aws-cdk/blob/34c7b9e7d882cd76a2b2f513781f9202385f4d30/packages/aws-cdk/lib/api/cloudformation-deployments.ts#L402 (it seems).

I've looked also into --exclude-build, but it seems it is not used with aws-lambda-nodejs...

Expected Behavior

Do not build & publish assets, which haven't changed.

Current Behavior

Assets are being build and published.

Reproduction Steps

Use aws-lambda-nodejs and deploy stack with Lambda(s).

Possible Solution

No response

Additional Information/Context

No response

CDK CLI Version

2.43.0

Framework Version

No response

Node.js Version

16.13.0

OS

MacOS 12.2.1

Language

Typescript

Language Version

TypeScript (4.7.4)

Other information

No response

[nick]

+1 - this would save a huge amount of time spend re-building files that haven't changed

[nick]

There might be a way to solve this by manually specifying NodejsFunctionProps.bundling.assetHash and ensuring the assetHash is updated whenever a relevant file changes. Will report back if I find a simple-ish solution.

[rix0rrr]

How much time does it take?

[github-actions[bot]]

This issue has not received a response in a while. If you want to keep this issue open, please leave a comment below and auto-close will be canceled.

[piotrmoszkowicz]

I have added assetHashes and this operation still is there... @rix0rrr I will give you ~time & ~count of lambdas in our case.

[piotrmoszkowicz]

But... If we avoid deployment & building (which means: zipping lambda code, sending new lambda code to AWS & deploying new lambda version) it would safe ~400-450 sec from our deployment time for sure (~70 lambdas).

[piotrmoszkowicz]

Is there any chance to work on that @rix0rrr. I would love to help, but I am not that educated in CDK from behind the scenes, so I am uncertain how we shall proceed with that one.

[tvb]

I see the same for the aws-lambda-python-alpha construct. No source file changes, still seeing a bundling and publishing. For me it would take out 3:37 minutes out of a 5:20 total pipeline run.

cdk deploy --require-approval never
Step 1/10 : ARG IMAGE=public.ecr.aws/sam/build-python3.7
Step 2/10 : FROM $IMAGE
latest: Pulling from sam/build-python3.9
e86b34a791fa: Pulling fs layer
9ba3f71dde1f: Pulling fs layer
c09bfc1b1fd5: Pulling fs layer
...
[0%] start: Building 5f67ddad2d5185486f13b5862b7730f02d21b7e9b8cbxxxxx:current_account-eu-west-1
[0%] start: Publishing 5f67ddad2d5185486f13b5868997730f02d21b7e9bxxxxxx:current_account-eu-west-1
[100%] success: Published 0fa75f8b14bfa78ef0ee43368c9d0ea7580c7429e4eafa9fexxxxx:current_account-eu-west-1

[rix0rrr]

Unfortunately, the current behavior is unavoidable the way bundling is currently being done.

The bundling build happens during cdk synth, and synth must happen before any other processing can happen. In a pipeline, there is no working directory, so there is no possibility of reusing previously built work.

I am aware that the current bundling story is not great, and have some ideas for improvements, but for now this is what it is, unfortunately.

I'm thinking that caching the cdk.out directory to S3 might provide some options for cache reuse... so you could try looking into that.

[piotrmoszkowicz]

Hmm, what do you exactly mean by caching cdk.out? Currently I have cdk.out on my machine and that long deploy time happens anyway (I assure you that most of lambdas inside that cache MUST BE exact same). So that cdk.out seem not to work either at least in my case...