getlift / lift

Expanding Serverless Framework beyond functions using the AWS CDK
MIT License
912 stars 111 forks source link

Add max concurrency support for AWS Lambda functions when using AWS SQS as an event source #294

Closed htxiong closed 1 year ago

htxiong commented 1 year ago

Use-case description

12 Jan 2023 AWS introduced maximum concurrency of AWS Lambda functions when using Amazon SQS as an event source (see release announcement here and compute blog from AWS)

And Serverless started to support it around 3 weeks. PR

I suggest we should define max concurrency in Lift.

Example Config

    alarm: engineering@xxx.com
    batchSize: 1
    maxRetries: 5
    maxConcurrency: 10
    worker:

Implementation Idea

Add the configuration for maximumConcurrency and modify Queue.ts. ( e.g. after this line)

this.configuration.worker.events = [
            // Subscribe the worker to the SQS queue
            {
                sqs: {
                    arn: this.queue.queueArn,
                    batchSize: batchSize,
                    maximumBatchingWindow: maximumBatchingWindow,
                    maximumConcurrency: maximumConcurrency
                    functionResponseType: "ReportBatchItemFailures",
                },
            },
        ];
mnapoli commented 1 year ago

That is 100% a good idea, thanks for opening this!

I'm wondering if maximumConcurrency shouldn't go one indentation lower, not in the worker but on the "queue" configuration itself, what do you think?

htxiong commented 1 year ago

Yaaas, I will work on a PR.

htxiong commented 1 year ago

@mnapoli can you review this PR for me? https://github.com/getlift/lift/pull/299

fredericbarthelet commented 1 year ago

Released in 1.25.0