search

logstash-plugins / logstash-output-google_cloud_storage

Apache License 2.0
9 stars 26 forks source link

Add option to perform upload in main event processing thread #16

Closed wfhartford closed 6 years ago

wfhartford commented 7 years ago

This is a work-around for the issues described in #15 which uploads files to GCS in the main event processing thread, and uploads the last part in the close method.

Behaviour should be entirely unchanged unless the new configuration option is specified"

  google_cloud_storage {
    upload_synchronous => true
  }

This ensures that if logstash exits gracefully, all received events will be uploaded to GCS.

josephlewis42 commented 6 years ago

@wfhartford thanks for the improvement! If you resolve conflicts with master I'd be happy to try this out myself and merge it.

wfhartford commented 6 years ago

@josephlewis42 I've resolved the merge conflicts and tested with my original use case.

josephlewis42 commented 6 years ago

@wfhartford, awesome! Thanks for taking the time to pick this back up again!