rackerlabs / lambda-uploader

Helps package and upload Python lambda functions to AWS
Apache License 2.0
270 stars 56 forks source link

Add functionality for Kinesis subscriptions #136

Closed dsouzajude closed 7 years ago

dsouzajude commented 7 years ago

This PR adds functionality for the lambda to subscribe to a Kinesis stream. The configuration is defined in the field subscription as follows:

{
  "name": "myFunc",
  "description": "myfunc",
  "region": "us-east-1",
  "handler": "function.lambda_handler",
  "role": "arn:aws:iam::00000000000:role/lambda_basic_execution",
  "requirements": ["Jinja2==2.8"],
  "ignore": [
    "circle.yml",
    ".git",
    "/*.pyc"
  ],
  "timeout": 30,
  "memory": 512,
  "subscription": {
    "kinesis": {
      "stream": "arn:aws:kinesis:eu-west-1:000000000000:stream/services",
      "batch_size": 10
    }
  },
  "vpc": {
    "subnets": [
      "subnet-00000000"
    ],
    "security_groups": [
      "sg-00000000"
    ]
  }
}

It also allows for the possibility to add other kinds of subscriptions, for example AWS SNS.

My use-case:

I'd like my lambda to process logs from a Kinesis stream, transform it into events that can be sent to Riemann for monitoring purposes. Therefore, after creating and uploading the lambda function, I need it to subscribe onto the stream to receive incoming data and process them.

jarosser06 commented 7 years ago

I made a comment but everything else looks good, once my question gets answered I think we can :shipit: .