Add logging customization script for Segment. Closes #2.

[ghost]

Provides the ability for users of this modular input to create custom data transforms before data is loaded into Splunk. Includes example using Segment payloads and in accordance with Splunk Logging Best Practices.

@john-paul-keenan /cc @treeder for review. Note this is my first time writing Python and my first time working with Splunk Modular Inputs.

[treeder]

@ulandj can you review please?

[ulandj]

hi @jhabdas , Your transform_data method is intended only for the Segment. for others it will not work unfortunately...

the main purpose of this plugin is to create a bridge between IronMQ and splunk enterprise framework. i.e. just get message from IronMQ queue and pass it to Splunk. we can't parse a message body as users need. because message bodies are different(may contain json, xml, string etc.) and it is difficult to make a general template for this. if you need an another message format feel free to edit the source code and use it.

Thanks

[ghost]

Thanks for taking a look, Ulan. You saw the script is disabled by default and requires the manual configuration step added to the README, correct? As a result, there would be no side effects or breaking changes for existing users. What all users would gain, however, is a clear way to deal with #2 when they encounter it. And given the constraints of the Splunk UI I see no other immediate solve for the problem at hand. Do you?

[ghost]

I guess it doesn't really matter. The breadcrumb is here now for others to follow. On my end I'm no longer confident our Segment Iron.io integration was a good choice over writing directly to SNS or the like and finding a way to leverage the AWS Splunk add-on instead.

Thanks to the Iron team for the help they've provided, but maintaining a custom modular input feels like a hack and necessitates a new direction and additional work.