search

influxdata / telegraf

Agent for collecting, processing, aggregating, and writing metrics, logs, and other arbitrary data.
https://influxdata.com/telegraf
MIT License
14.59k stars 5.56k forks source link

S3 Plugin #6617

Open robertkeithrippetoe opened 4 years ago

robertkeithrippetoe commented 4 years ago

Feature Request

Opening a feature request kicks off a discussion.

Proposal:

Input Plugin to pull data using a file-reader pattern from s3 buckets into telegraph.

Current behavior:

Telegraf cannot natively be configured to consume specific formats of s3 data.

Desired behavior:

Telegraf can utilize a configurable s3 plugin in order to to consume specific formats of s3 data.

Use case:

Configure an s3 plugin to consume and parse data from AWS loadbalancer logs stored in an S3 bucket to feed telegraf metrics such as the number of unsuccessful http requests made to a specific loadbalancer.

Next Steps: If there are no objections, I will open a pull request to start work on supporting this new plugin.

danielnelson commented 4 years ago

Sounds good, would be helpful if you can add the proposed TOML config here as soon as you have it for discussion.

safuente commented 4 years ago

Any progress on this feature? I think it could be very useful

NoxInmortus commented 3 years ago

Would also be interrested by this input plugin

JimHagan commented 3 years ago

New Relic would be interested in this plugin. Be curious to know if anyone has implementation proposals or sample TOML files.

pmathur231 commented 1 year ago

Dell is interested in getting this plugin too. Any update on this one ? if anyone has implemented or has a workaround , appreciate if you can share

Hipska commented 1 year ago

You could mount the S3 bucket and then use the file, tail or directory_monitor plugins..