search

aws / aws-cdk

The AWS Cloud Development Kit is a framework for defining cloud infrastructure in code
https://aws.amazon.com/cdk
Apache License 2.0
11.71k stars 3.94k forks source link

glue: support hotswap for job code assets #21726

Open humanzz opened 2 years ago

humanzz commented 2 years ago

Describe the feature

Glue Job make use of several Code assets as can be seen in

The feature requested is to allow using cdk deploy --hotswap <stack containing glue job with only code assets change> to bypass doing a full cfn deployment in favour of updating the code assets only.

Use Case

For personal deployments, Glue jobs - like Lambda functions - most frequent change is changes to the job's code assets. Like Lambda, the ability to short-circuit a full cfn deployment and to change the code assets only can accelerate personal deployments

Proposed Solution

No response

Other Information

No response

Acknowledgements

CDK version used

2.38.1

Environment details (OS name and version, etc.)

macOS

tmokmss commented 2 years ago

As a workaround I often use BucketDeployment construct to deploy code for glue job and refer to the file by glue.Code.fromBucket function. BucketDeployment does support hotswap :)

kaizencc commented 2 years ago

I labeled this effort medium because it will require integ tests to get this right, but I'd be happy to get a PR from the community on this.

humanzz commented 2 years ago

@kaizencc any high level guidance on what it takes to have that supported?

I can try to give it a go at a PR but I know nothing about how hotswap is implemented

humanzz commented 2 years ago

@kaizencc I've checked https://github.com/aws/aws-cdk/blob/main/packages/aws-cdk/lib/api/hotswap/lambda-functions.ts for inspiration and I see some challenging aspects