Open dambaron opened 1 year ago
peterwoodworth
commented
1 year ago I'm not noticing any difference @dambaron, the time seems to remain about the same for me. Could you post reproduction info like your nodejsfunction definition, as well as the actual handler file? Thanks
dambaron
commented
1 year ago Hi @peterwoodworth, I won't be able to post the handler itself but the definition goes like this
const nodejsFunction = new NodejsFunction(scope, `${lambdaReference.name}Function`, {
depsLockFilePath: join(__dirname, '../yarn.lock'),
runtime: Runtime.NODEJS_18_X,
tracing: Tracing.ACTIVE,
timeout: Duration.seconds(10),
memorySize: 256,
bundling: {
minify: false,
esbuildArgs: {
'--main-fields': 'module,main',
},
externalModules: []
},
// truncated
})I must add that what we are experiencing happends during a hotswap with a command like:
yarn run cdk deploy -e my-stack --hotswap --profile my-sso-profile
I'm also attaching to CPU profiles I made for the same command using v2.51.1 then v2.63.0
What I see is that the bundling seems to work fine but then a very long idle time happens.
peterwoodworth
commented
1 year ago thanks for the additional info,
I'm also curious if you notice anything during the idle time you mention if you use the -v option, could you try this?
dambaron
commented
1 year ago thanks for the additional info,
I'm also curious if you notice anything during the idle time you mention if you use the
-voption, could you try this?
Here you are, I cranked the logs all the way up but the outputs are quite similar. cdk.deploy.hotswap.v2.63.0.log cdk.deploy.hotswap.v2.55.1.log
dambaron
commented
1 year ago @peterwoodworth any news after I provided you the logs ?
dambaron
commented
1 year ago @peterwoodworth uping this thread once again
peterwoodworth
commented
1 year ago Thanks for sharing your logs, and sorry for the delay here @dambaron
Looking at the logs, it doesn't seem to show anything helpful. I'm curious, at this point in the logs:
BLOCK A
Bundling asset oxp-dbn-api-engagement-lambda-stack/HandleInteractionsFunction/Code/Stage...
$ /Users/dbaron/Projects/voila/backend/oxp-api/node_modules/.bin/esbuild --bundle /Users/dbaron/Projects/voila/backend/oxp-api/engagement/src/interaction/InteractionHandler.ts --target=node18 --platform=node --outfile=/Users/dbaron/Projects/voila/backend/oxp-api/cdk.out/bundling-temp-6cdfe7655c7745338ff0bb31f7835257d757245287d91c0efb5b492bd62c1856/index.js --main-fields=module,main
...c7745338ff0bb31f7835257d757245287d91c0efb5b492bd62c1856/index.js 925.3kb
BLOCK B
Bundling asset oxp-dbn-api-engagement-lambda-stack/IncrementEngagementsFunction/Code/Stage...
$ /Users/dbaron/Projects/voila/backend/oxp-api/node_modules/.bin/esbuild --bundle /Users/dbaron/Projects/voila/backend/oxp-api/engagement/src/engagement/EngagementHandler.ts --target=node18 --platform=node --outfile=/Users/dbaron/Projects/voila/backend/oxp-api/cdk.out/bundling-temp-4ff2215903da0442bafd5ffef7e2ce9ab0ecec693dc945c995cac9c68c14d5a1/index.js --main-fields=module,main
...3da0442bafd5ffef7e2ce9ab0ecec693dc945c995cac9c68c14d5a1/index.js 996.8kb
BLOCK C
✨ Synthesis time: 65.75sdoes the "very long idle time" you describe occur in Block A, B, or C? Based on everything I'd guess C, is this right?
dambaron
commented
1 year ago Exact @peterwoodworth , the slowdown occurs in block C. I've been able to retrieve the esbuild commands and run them outside the deploy phase and duration is the same inside or outside cdk. What I'm experiencing is awfully close to this (https://github.com/aws/aws-cdk/issues/19021)
peterwoodworth
commented
1 year ago If it's similar, could you check CloudTrail to see if a bunch of duplicate calls are going off?
dambaron
commented
1 year ago @peterwoodworth CloudTrail was not enabled but through an http proxy I was able to determine that I have the same number of requests with roughly the same execution times.
The only thing I see now is that a significant amount of time is spent here in the selectStacksForDeploy function.
Describe the bug
When bumping aws-cdk from
2.55.1to2.56.1my team witnessed a significant slowdown during the esbuild bundling. The bundling increased more than 3x. This behavior has been observed all the way up to2.62.2.Expected Behavior
Bundling time should remain consistent between versions (or decrease).
Current Behavior
Bundling time has been multiplied by 3 only by bumping the aws-cdk version in the package.json.
Reproduction Steps
2.51.1.2.61.1. 3.Examine synthesis timeIn my case:
2.55.1: 12.02s2.56.1: 65.07sPossible Solution
No response
Additional Information/Context
package.json
CDK CLI Version
2.56.1
Framework Version
No response
Node.js Version
v18.12.1
OS
Mac OS 13
Language
Typescript
Language Version
4.9.4
Other information
No response