Support for nested lambda function deployment (packaging and upload to s3) as part of CF template

aws-solutions / aws-control-tower-customizations

The Customizations for AWS Control Tower solution combines AWS Control Tower and other highly-available, trusted AWS services to help customers more quickly set up a secure, multi-account AWS environment using AWS best practices.

https://docs.aws.amazon.com/controltower/latest/userguide/cfct-overview.html

Apache License 2.0

356 stars 205 forks source link

Support for nested lambda function deployment (packaging and upload to s3) as part of CF template #82

Open rohit3d2003 opened 2 years ago

rohit3d2003 commented 2 years ago

Is your feature request related to a problem? Please describe.

We have a lot of custom AWS Config rules backed by custom lambdas. These lambda functions are written in python but instead of writing it inline inside CloudFormation template, we have a need to define them in folder and be able to package them in s3 and dynamically deploy as part of cloud-formation. Currently manifest file does not support 'aws cloudformation package' command. If there are other options, please let me know. Our goal is to use CT without customizing the AWS provided solution unless required

Describe the feature you'd like

Support for lambda packaging as part of manifest definition or something along these lines

Additional context

deolank commented 2 years ago

Thanks for your feedback. We have added this to our backlog.

cacack commented 2 years ago

We worked around this by modifying our pipeline to include a "preprocessor" stage which takes care of creating the Lambda zips based on directories in the repo..

But I'd prefer to not have deviated.

ryanash999 commented 1 year ago

@cacack

We worked around this by modifying our pipeline to include a "preprocessor" stage which takes care of creating the Lambda zips based on directories in the repo..

But I'd prefer to not have deviated.

Can you provide details how you hooked the preprocessor into the existing CodePipeline?

rjenks commented 1 year ago

Not sure how cacack managed this, but one way is to:

Set the customization pipeline to trigger from S3 instead of Git
Create a pre-processor pipeline that runs prior to the customization pipeline that triggers from Git and outputs a zip to S3 to trigger the customization pipeline. Do anything you need to in this pipeline.

cacack commented 1 year ago

@ryanash999 our solution was something like this:

Created an S3 bucket to house the Lambda zips
Located all of our Lambda functions into unique directories housed in a subdirectory in the repo.
Created a CodeBuild project which essentially walked the directories, zipping them individually and uploading to S3.
Modified the CfCT CodePipeline to call our Preprocess stage before the Build stage.
Updated our CfCT templates to reference the S3 URL for the given Lambda.

mdaehnert commented 1 year ago

We saw same need for more deployment capabilities and built a solution on top of CfCT with support for AWS SAM. It means you can now easily develop with SAM features like packaged lambdas and deploy them via CfCT. Feel free to have a look at our AWS Blog Post and GitHub repo for more details.