search

aws / copilot-cli

The AWS Copilot CLI is a tool for developers to build, release and operate production ready containerized applications on AWS App Runner or Amazon ECS on AWS Fargate.
https://aws.github.io/copilot-cli/
Apache License 2.0
3.49k stars 404 forks source link

Make "building" lambdas easier #3972

Open sethtjf opened 2 years ago

sethtjf commented 2 years ago

I've been searching for a way to "build" a lambda function with copilot. The addon docs and related issues like #3659 outline a few options as far as I can tell:

  1. Download dependencies locally into the directory containing the lambda code so that it can be zipped and deployed
  2. Using build scripts or a Makefile that builds the lambda package and then runs copilot deploy
  3. As mentioned in #3659, using SAM+Copilot to create a "build" concept before deployment
  4. Adding the lambda function to the primary docker container so that it's built by copilot and then change the container command

Ideally, building lambda would get first-party treatment by copilot. There may be limitations why this can't be the case, but I would love to see one of two solutions:

  1. Support for build options inside the addon file so that the lambda (or even another docker container) is built and deployed as part of the regular copilot deploy and in pipeline; or
  2. Support for Worker Services as lambda functions so that code can be easily packaged and built as part of deployment and pipelines

Would love to hear what the team is thinking about doing here. I think there is a huge opportunity for copilot + lambda.

Lou1415926 commented 2 years ago

Hello @sethtjf ! Thank you very much for the feature request! Just a few follow up / clarification questions for us to kickstart a discussion:

  1. Could you elaborate on what you meant by "build"? Is it mostly referring to building the image that's used by the lambda function?
  2. Do you have any specific use case in mind? How do you envision your lambda to work with the other services?

Other folks with similar requests are welcome to chime in!

Thank you!

sethtjf commented 2 years ago

Hey @Lou1415926, thanks for following up. I'm happy to kick of a discussion:

  1. I think that's right. Ideally I would be able to define a lambda function as an addon and specify a Dockerfile that is built, deployed to ECR, then used to create a lambda function.
  2. Use case is to use lambda to create something similar to the Worker Service pattern linked above. I'd prefer to use lambda functions so that I can publish to SNS topics, subscribe with SQS and have my lambda workers do their thing. A reason for tighter integration with lambda is to more easily allow my lambda to talk to the the EFS and RDS services that I have copilot provision. I've spend a fair bit of time recently trying to get everything to talk to each other properly and it's not been easy (nor can I confirm that it works yet). So anything that makes that flow easier and allows me to replicate across different environments would help. Especially as I look to add more worker functions in the future.

I hope that helps to clarify!

efekarakus commented 2 years ago

Just sharing some interesting links to explore to make building Lambda functions easier:

oxbits commented 2 weeks ago

are people able to use AWS copilot and SAM together effectively ?