Add relative S3 path to events

[erikwebb]

I have a use case where the relative path to the file within S3 provides some context. This adds the S3 path as a field to each event.

[elasticsearch-release]

Jenkins standing by to test this. If you aren't a maintainer, you can ignore this comment. Someone with commit access, please review this and clear it for Jenkins to run; then say 'jenkins, test it'.

[elasticsearch-release]

Jenkins standing by to test this. If you aren't a maintainer, you can ignore this comment. Someone with commit access, please review this and clear it for Jenkins to run; then say 'jenkins, test it'.

[DrValani]

I would suggest using "s3key" instead of "path".

"path" can then be the fullpath, that is "bucket_name + key"

Yogi

[ph]

I suggest we do the following we save that information in the @metadata hash and under a top s3 key? This will make it backward compatible since we don't serialize from the @metadata by default but people can pick it with the mutate filter.

In the ruby code this would look like this.

event["[@metadata][s3]"] = { "key" => key }

Also test would be awesome so we dont' break this feature in a future release.

[todd534]

This seems like a highly-desired feature, with multiple tickets asking for it and a couple entries on Stack Overflow asking how to do the thing, but the PR seems to have stagnated. Is there any way that I could help out? Happy to make @ph's suggested change and resolve merge conflicts, but not sure what the customs are with respect to updating a branch on someone's fork.

[kaisecheng]

A similar feature has merged, hence close this