search

amazon-archives / logstash-input-dynamodb

This input plugin for Logstash scans a specified DynamoDB table and then reads changes to a DynamoDB table from the associated DynamoDB Stream.This gem is a Logstash plugin required to be installed on top of the Logstash core pipeline. This gem is not a stand-alone program.
Apache License 2.0
105 stars 27 forks source link

LS2 support + safe stop? (shutdown refactor) #22

Open teebu opened 8 years ago

teebu commented 8 years ago

Thanks to @mantika I was able to install dynamodb plugin to LS2, but when running a test, I can't seem to get it to stop (ctrl+c) it just keeps going and throws {"inflight_count"=>0 ... } warnings. It does stop in LS1.5.

Also, sudo service logstash stop it never wants to stop. I have to kill -9 the pid.

I belive this maybe related to new plugin API, some ref to a request here https://github.com/logstash-plugins/logstash-input-dynamodb/issues/3

https://github.com/elastic/logstash/issues/3963

Looking in the logs:

{:timestamp=>"2016-08-18T22:54:55.141000+0000", :message=>"SIGTERM received. Shutting down the agent.", :level=>:warn}
{:timestamp=>"2016-08-18T22:59:29.088000+0000", :message=>#<LogStash::PipelineReporter::Snapshot:0x3cc4fe9f @data={:events_filtered=>6604, :events_consumed=>6604, :worker_count=>1, :inflight_count=>75, :worker_states=>[{:status=>"run", :alive=>true, :index=>0, :inflight_count=>75}],
{:timestamp=>"2016-08-18T23:01:37.979000+0000", :message=>"The shutdown process appears to be stalled due to busy or blocked plugins. Check the logs for more information.", :level=>:error}

And then it just continues. It may never end because it's always trying to fill the buffer?