search

logstash-plugins / logstash-input-s3

Apache License 2.0
57 stars 150 forks source link

Add setting to skip the sincedb timestamp check #192

Closed SpencerLN closed 3 years ago

SpencerLN commented 4 years ago

Adding a new setting that allows a user to configure the sincedb timestamp check to be skipped. This is useful when the delete mode is in enabled and resolves issues where multiple files with the same last modified timestamp are currently not processed in a reliable manner (#57 #191).

The sincedb_ignore setting should not be set to true unless delete is also set to true, otherwise the same files could be repeatedly process.

Thanks for contributing to Logstash! If you haven't already signed our CLA, here's a handy link: https://www.elastic.co/contributor-agreement/

kaisecheng commented 3 years ago

Hello @SpencerLN Thanks for submitting the PR to get over the missing file ingestion problem. The v3.6.0 fixed the issue, hence close this