There are three ways that have come up in which we'd like streambot to be able to feed lambda functions: SNS, Kinesis, DynamoDB Streams. Each represent slightly different scenarios, and the differences can be difficult (but not impossible) to capture in a single CloudFormation template.
First of all, let's remove the concept of streambot creating the event source for you, and instead assume that you will always provide the ARN of your source. In the current, Kinesis case this would mean:
Next I can see two approaches to adjusting streambot's CloudFormation template:
A single template: Can we determine the nature of the event source (SNS, Kinesis, DynamoDB stream) within the template based on nothing but the ARN that is provided as a parameter? If so, can this guide the creation of appropriate IAM roles and outputs for that type of event source?
Distinct templates: We could have streambot provide three templates, each specific to a type of event source. We have one for Kinesis, we could clone it and adjust it for DynamoDB streams before moving on to SNS-triggered functions.
There are three ways that have come up in which we'd like streambot to be able to feed lambda functions: SNS, Kinesis, DynamoDB Streams. Each represent slightly different scenarios, and the differences can be difficult (but not impossible) to capture in a single CloudFormation template.
First of all, let's remove the concept of streambot creating the event source for you, and instead assume that you will always provide the ARN of your source. In the current, Kinesis case this would mean:
Next I can see two approaches to adjusting streambot's CloudFormation template:
A single template: Can we determine the nature of the event source (SNS, Kinesis, DynamoDB stream) within the template based on nothing but the ARN that is provided as a parameter? If so, can this guide the creation of appropriate IAM roles and outputs for that type of event source?
Distinct templates: We could have streambot provide three templates, each specific to a type of event source. We have one for Kinesis, we could clone it and adjust it for DynamoDB streams before moving on to SNS-triggered functions.
cc @jakepruitt and also @freenerd -- if this pans out it may supersede your https://github.com/mapbox/streambot/pull/16