search

splunk / splunk-aws-project-trumpet

MIT License
95 stars 31 forks source link

Feature Request: Add Support for AWS CloudWatch Metrics, Description, and S3 Access Logs. #1

Open connor-tyndall opened 5 years ago

connor-tyndall commented 5 years ago

Currently, we have to bring in AWS CloudWatch Metrics, Description, and S3 Access Logs via modular inputs using EC2 Heavy Forwarders via the Splunk Add-on for AWS. Although this works, it provides challenges in terms of scale, whereas we are looking to use "serverless" methods rather than long-running instances.

https://docs.splunk.com/Documentation/AddOns/released/AWS/CloudWatch https://docs.splunk.com/Documentation/AddOns/released/AWS/DescriptionInput https://docs.splunk.com/Documentation/AddOns/released/AWS/SQS-basedS3

nstonesplunk commented 5 years ago

Hi @connor-tyndall

Thanks for the request!

From previous investigation, implementing AWS Description and AWS CloudWatch Metrics will require pulling on an interval and some form of persisted store for error handling and guaranteed eventual delivery. We have prototyped this but ran into issues/concerns with Lambda timeouts and the additional DynamoDB infrastructure. This is still something we are iterating on since it really is one of the main missing pieces to populate the Splunk App for AWS serverlessly.

s3 access logs are also tricky but can follow the same path as AWS Config snapshots do now (s3 -> Lambda -> Splunk) or perhaps route through s3 -> Lambda -> CloudWatch logs -> Firehose -> Splunk. I will look into this further.

connor-tyndall commented 5 years ago

@nstonesplunk Any update on this?

connor-tyndall commented 4 years ago

@nstonesplunk Any update?

gliptak commented 4 years ago

@nstonesplunk I would like to learn more about your thoughts above

"From previous investigation, implementing AWS Description and AWS CloudWatch Metrics will require pulling on an interval and some form of persisted store for error handling and guaranteed eventual delivery. We have prototyped this but ran into issues/concerns with Lambda timeouts and the additional DynamoDB infrastructure."

While I understand that log delivery is expected to be guaranteed, could (Description and CloudWatch) metrics be approached on best effort basis?

kerryhatcher commented 3 years ago

@nstonesplunk Lambda's timeouts and other features have changed a lot in the last 2 years. Is there anything any of us can do to help test or prototype possible solutions?

kerryhatcher commented 3 years ago

@nstonesplunk Here is Loggly's open source implementation. Perhaps it could be forked or used in some way...

https://github.com/varshneyjayant/cloudwatch-metrics-to-loggly/blob/master/index.js