logstash-plugins / logstash-input-s3

Apache License 2.0
57 stars 150 forks source link

Unable to ingest files restored from Glacier #199

Closed magical-travvo closed 4 years ago

magical-travvo commented 4 years ago

When trying to ingest files from S3 that have been temporarily restored from Glacier, they are ignored by this input, as files are excluded based on being in the "Glacier" storage class.

Restored files still remain in a "Glacier" storage class, but include additional metadata indicating that they are temporarily available. We regularly archive data processed by Logstash to S3 using the logstash-s3-output plugin, and it would be helpful to be able to easily reingest that data once a restore has been completed.

magical-travvo commented 4 years ago

I've raised https://github.com/logstash-plugins/logstash-input-s3/pull/197 to address the issue. I'm not sure why the builds are failing. The tests all pass locally.

kares commented 4 years ago

failures are failing due a new major (2.0) of the devutils gem. we'll be addressing the issue shortly (on master) ... you should be able to rebase afterwards.

kares commented 4 years ago

fix delivered at https://github.com/logstash-plugins/logstash-input-s3/pull/197 and available since 3.5.0