logstash-plugins / logstash-input-s3

Apache License 2.0
57 stars 150 forks source link

Add max_file_age option #178

Closed alexjurkiewicz closed 2 years ago

alexjurkiewicz commented 5 years ago

Simple way to exclude old files. Example use case: a bucket which holds daily log files, but you only want to keep the most recent n days.

It's possible to drop lines from old log files further into the processing pipeline, but skipping the files entirely is significantly more efficient.

Thanks for contributing to Logstash! If you haven't already signed our CLA, here's a handy link: https://www.elastic.co/contributor-agreement/