awslabs / aws-solutions-constructs

The AWS Solutions Constructs Library is an open-source extension of the AWS Cloud Development Kit (AWS CDK) that provides multi-service, well-architected patterns for quickly defining solutions
https://docs.aws.amazon.com/solutions/latest/constructs/
Apache License 2.0
1.22k stars 247 forks source link

aws-s3-eventbridge #494

Open knihit opened 2 years ago

knihit commented 2 years ago

Similar to receiving S3 notifications on SQS, SNS, or Step functions, S3 event notifications can also be sent to Amazon Event Bridge. Event Bridge provides some advantages over SQS and SNS as detailed in this blog post https://aws.amazon.com/blogs/compute/using-dynamic-amazon-s3-event-handling-with-amazon-eventbridge/

Use Case

I am plan to use this construct in my solution to build S3 PutObject event notification.

Proposed Solution

The solution will be very similar to what is proposed in the blog post https://aws.amazon.com/blogs/compute/using-dynamic-amazon-s3-event-handling-with-amazon-eventbridge/ using AWS CloudTrail

Other


This is a :rocket: Feature Request

biffgaut commented 2 years ago

We like this idea - we actually already have most of it in place. aws-s3-stepfunctions goes thru EventBridge, so we can create this just by extracting the s3 to eventbridge portion of that construct. We've added that task to our roadmap (no date promised :-)

biffgaut commented 2 years ago

After looking deeper into this, an aws-s3-eventbridge construct targeting the default event bus wouldn't do anything, since those events are already sent to the default bus. If we allow a custom event bus, then it adds value - but the value is all within EventBridge. It seems to us a better construct to write would be aws-eventbridge-eventbridge, that allows customers to move event from the default bus to a custom bus. This would enable sending S3 events to a custom bus, but also provide the same functionality for all the AWS services that send events to the default bus. The alternative would be createing aws-sqs-eventbridge, aws-ec2-eventbridge, etc.

What we lose in doing it this way rather than with aws-s3-eventbridge is the option to automatically create a CloudTrail. But setting up CloudTrail in every account is a significant best practice - hopefully every account has set this up. And if an account does not have it set up, perhaps not including it in this construct will encourage the account owner to do so in a way less transient than as part of a dynamically created and destroyed construct.

biffgaut commented 2 years ago

This discussion is impacted by this announcement. CloudTrail is no longer needed for S3 events. An aws-s3-eventbridge construct at this point would only allow clients to send S3 events to a custom event bus.

biffgaut commented 2 years ago

What this construct wants to do is being done by enabling EventBridge for the S3 bucket, so as described this functionality is now available without this implementation. This opens up a discussion of how to link new services to be triggered by EventBridge events from S3 using constructs, and also how to get S3 notification events to an custom Event Bus.

Leaving this open for discussion and further thought.

knihit commented 2 years ago

Enabling eventbridge is a property within the S3 bucket. Its the rules and the patterns that need to be defined. Note that, this only allows sending events to the default bus. (Any custom bus targets would still require additional rules configuration and defining targets).

biffgaut commented 2 years ago

We're thinking about an aws-eventbridge-eventbridge construct to take and event source feeding the default bus and forward them to a custom bus. It would take an event source, such as an S3 bucket, as a prop. But event sources are tricky, lambda has it's own CDK library for responding to various events.

We've also opened an issue on the CDK repo to ad enabling eventbridge to the L2 S3 object (our recent release does it by directing setting the CloudFormation on the underlying L1 construct). No response as yet.