cherweg / logstash-input-s3-sns-sqs

logstash input downloading files from s3 Bucket by OjectKey from SNS/SQS
Other
29 stars 35 forks source link

Keep S3 path name as field #12

Closed danielkasen closed 6 years ago

danielkasen commented 6 years ago

Not sure if I missed this in the code, but is there a way we could include the S3 path as a field that get's added to the document? I'm looking to leverage the filename as an indexed field but looks like it gets stripped out by the input plugin.

cherweg commented 6 years ago

The S3 path is only in available as logstash metadata. You could create a field with mutate - add field.

Regards Christian

Von meinem iPhone gesendet

Am 21.08.2018 um 23:50 schrieb Daniel Kasen notifications@github.com:

Not sure if I missed this in the code, but is there a way we could include the S3 path as a field that get's added to the document? I'm looking to leverage the filename as an indexed field but looks like it gets stripped out by the input plugin.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

danielkasen commented 6 years ago

Awesome, thanks!

danielkasen commented 6 years ago

For those that might want to do the same: mutate { add_field => { "s3_path" => "s3://%{[@metadata][s3][bucket_name]}/%{[@metadata][s3][object_key]}" }}