CoorpAcademy / serverless-plugins

Collection of serverless plugins :zap:
230 stars 131 forks source link

Serverless offline sqs stop trigger event #147

Open drizzef opened 4 years ago

drizzef commented 4 years ago

Hi,

We are working with a serverless offline SQS plugin and we didn't notice what we did but in offline mode, the lambda stops triggering when there is a message on the queue(we are using Elasicmq on the local environment).

this is the YAML file:

```
app: xxxxx-micro-services
service: microservices
provider:
  name: aws
  runtime: nodejs12.x
  stage: ${opt:stage, 'dev'}
  region: eu-west-1
  iamRoleStatements:
    - Effect: "Allow"
      Action:
        - "sqs:SendMessage"
        - "sqs:GetQueueUrl"
      Resource:
        Fn::GetAtt:
          - EmailsQueue
          - Arn
    - Effect: "Allow"
      Action:
        - "sqs:ListQueues"
      Resource:
        Fn::GetAtt:
          - EmailsQueue
          - Arn

functions:
  emailProducer:
    handler: src/handler.emailProducer
    events:
      - http:
          path: /email
          method: post
    environment:
      DEFAULT_FROM_EMAIL: ${self:custom.email.defaultAddress}
      EMAILS_QUEUE_URL: ${self:custom.queue.email.url}
      EMAILS_ACCESS_KEY_ID: ${self:custom.queue.email.accessKeyId}
      EMAILS_SECERT_ACCESS_KEY: ${self:custom.queue.email.secretAccessKey}
      EMAILS_REGION: ${self:custom.queue.email.region}
  emailConsumer:
    handler: src/handler.emailConsumer
    environment:
      NODE_ENV: ${self:custom.env}
      EMAIL_SMTP_USER_NAME: ${self:custom.email.smtp.userName}
      EMAIL_SMTP_PASSWORD: ${self:custom.email.smtp.password}
    events:
      - sqs:
          arn:
            Fn::GetAtt:
              - EmailsQueue
              - Arn

plugins:
  - serverless-plugin-typescript
  - serverless-offline
  - serverless-offline-sqs

resources:
  Resources:
    EmailsQueue:
      Type: AWS::SQS::Queue
      Properties:
        QueueName: emails

custom: ${file(${opt:stage, self:provider.stage, 'dev'}.config.yml)}

the dev.config.yaml, we include it in the custom block.

env: test email: smtp: userName: fdsfdsf@dasd.cc password: 123456 defaultAddress: fdfsdfsdf@cc.cc queue: email: region: eu-west-1 accessKeyId: root secretAccessKey: root url: http://localhost:9324/queue/emails serverless-offline: httpPort: 4400 serverless-offline-sqs: autoCreate: true # create queue if not exists apiVersion: "2012-11-05" endpoint: http://0.0.0.0:9324 region: eu-west-1 accessKeyId: root secretAccessKey: root skipCacheInvalidation: false

crajbanshi commented 3 years ago

I am also facing the same issue. Only one message is reading at the time of start, latter I am pushing data in the queue, but it does not trigger the serverless-ofline-sqs.

ebarault commented 3 years ago

hi @AdrieanKhisbe Same thing here, only one message is processed, then following messages don’t trigger the lambda function. This started when moving from serveless-offline 5 to 6+.

UPDATE: I tracked the issue down the code and discovered that the call to call to async functionlambdaFunction.runHandler() here https://github.com/CoorpAcademy/serverless-plugins/blob/master/packages/serverless-offline-sqs/src/sqs.js#L87 does not resolve if the lambda function handler returns None (pyhon).

After explicitly returning a value the lambdaFunction.runHandler() resolves and the plugin can handle a new sqs message event.

As returning None is allowed (cf. https://docs.aws.amazon.com/lambda/latest/dg/python-handler.htm) this is still a bug, but it can be overcame by returning an explicit value. Make sure also to return a value on errors, and not just raise/throw or it will also block.

I tracked the bug down in serverless-offline plugin and the async handler blocks here https://github.com/dherault/serverless-offline/blob/master/src/lambda/LambdaFunction.js#L269 when the lambda handler does not return truthy.

However, blocking there seems to be exclusive to the serverless-offline-sqs invoke, as it does not block for explicit lambda invoke. So telling whether the bug is in serverless-offline or serverless-offline-sqs is still to be decided.