search

ptdavies17 / CloudwatchFH2HEC

Cloudwatch Logs Transform for Firehose: formats into Splunk HEC Event
Other
14 stars 17 forks source link

Proper way to support non VPC and cloudtrail sourcetypes #1

Open rh46 opened 5 years ago

rh46 commented 5 years ago

RE: How to Ingest Any Log from AWS Cloudwatch Logs via Firehose

I was wondering how to best use CloudwatchFH2HEC.py to ship other log sourcestypes besides VPC and cloudtrail logs (the only two sourcestypes defined in the example script). Which of the approaches below would do you recommend if any? Ideally I could use the same transform function for all Firehose to HEC log shipping.

  1. Add a case statement to match additional cloudwatch log group names to their destination sourcetypes
  2. don't set the sourcetypes at all and let Splunk handle it somehow
  3. set SPLUNK_SOURCETYPE=aws:firehose:json

Alternatively I could create separate lambda functions for each sourcetype and pass different values for SPLUNK_SOURCETYPE in the environment variable configuration... but that feels like an anti-pattern.

List of example sourcetypes/use-cases from cloudwatch logs

gliptak commented 4 years ago

From above list, I submitted https://github.com/ptdavies17/CloudwatchFH2HEC/pull/3 for lambda support