Proposal: Add AWS EventBridge Output

[embano1]

I would like to contribute an eventbridge output/sink. EventBridge is very popular amongst the AWS communities to build event-driven architectures on AWS. Let me know if this is desired and I can work myself through the docs and contribution guidelines.

An MVP should support:

• a common schema for setting required EventBridge event envelope fields, such as source and detail-type (event type)
• batching
• transform non-JSON encoded messages to JSON (EventBridge only supports JSON-encoded input)
• optional: DLQ (SQS)
• optional: client-side filtering (sync EventBridge filtering rules locally to reduce ingestion volume)

Are there any specific guidelines I need to follow for such a cloud provider sink besides what is mentioned in the docs?

Disclosure: I work on the EventBridge service team.

[mihaitodor]

Hey @embano1, sure, PRs are welcome! You can find the contributor guidelines here and you'll probably want to copy one of the adaptors that's based on the public API, such as the Pulsar one. Your code should go in here and you'll need to add extra imports here if it ends up being a standalone subpackage in the aws one.

[embano1]

Thx @mihaitodor this is helpful! I'm planning to get some dedicated time for this in March. In terms of reference implementation, I'll probably go with SQS as it's closer to how EventBridge works.

Also, I assume I can only provide an Output implementation for now (as this is the common pattern I foresee users interacting with Benthos and EventBridge).

There seem to be two different Output implementations: a regular one OutPut and a BatchOutput. EventBridge supports both, so should I go with BatchOutput and provide some configuration option to influence the batching behavior, e.g. batchSize=1 to immediately fire?

[mihaitodor]

Sure, that makes sense. Feel free to scavenge whatever you need from the SQS code, but do note that the config stuff should be based on the public API instead of modules like github.com/benthosdev/benthos/v4/internal/docs, which is why I pointed out the Pulsar implementation as a reference above.

[embano1]

Thx for all the details! Perhaps I'll pencil in a Zoom with you some time in March (we're in close time zones) before spamming around too much here ,)

[mihaitodor]

You're welcome! Feel free to join one of the blobtalks on Discord. They take place every two weeks or so and Ash will be there too, since he's the maintainer.