search

Graylog2 / graylog-plugin-aws

Several bundled Graylog plugins to integrate with different AWS services like CloudTrail and FlowLogs.
Other
91 stars 37 forks source link

sns events to graylog #18

Open eyablonowitz opened 8 years ago

eyablonowitz commented 8 years ago

I would like to send SNS events to Graylog. This is a feature that is available in competitive products (e.g. https://www.loggly.com/docs/amazon-sns/).

It seems like this Plugin already has a lot of the pieces to make this happen. It can already poll an SQS queue subscribed to an SNS topic for Cloudtrail log notifications. But it doesn't appear to support the more generic case of simply wanting to send events from any SNS topic into Graylog.

Is there a way to make this work with graylog-plugin-aws as-is? If not, would it make sense to support the SNS->Graylog use case in this plugin?

wrsuarez commented 8 years ago

You could use a Lambda event to send the SNS payload to Graylog using the pygelf library. If all you want is the SNS message and don't need to use the payload to follow down a trail to another data element a Lambda event would do it. Even a simply Python script could read and parse the SNS message (as the JSON structure is fairly fixed) and send it as a GELF entry to Graylog.

eyablonowitz commented 8 years ago

Thanks @wrsuarez! I will have a look at that option.

But whether through graylog-plugin-aws or some other means (e.g. Lambda/pygelf), SNS to Graylog still seems like a use-case that should be well supported/documented as it is for Loggly (https://www.loggly.com/docs/amazon-sns/).