ph
commented
8 years ago I don't think this is an issue with the plugin, since MAX_MESSAGES_TO_FETCH is an internal magic number and we don't expose it to the user.
What is currently happening here is when you set the value to one, the library wont return a collection and the message itself doesn't quack like a enumerable.
They are doing some wise optimization ;)
Did you find why not all your messages were processed, what is an ordering issue?
When we work with SQS messages we do an ack to AWS when we have successfully sent the event down the pipeline. If SQS doesn't receive the ack it will resend the event eventually.
While troubleshooting an issue of why not all my SQS messages are being processed, I tried to set the MAX_MESSAGES_TO_FETCH to 1 and got the following.
{:timestamp=>"2015-10-14T23:20:06.700000+0000", :message=>"A plugin had an unrecoverable error. Will restart this plugin.\n Plugin: <LogStash::Inputs::SQS_S3 access_key_id=>\"###\", secret_access_key=>\"###\", queue=>\"###\", bucket=>\"###\", codec=>\"###\", debug=>false, threads=>1, region=>\"us-east-1\", delete=>false, interval=>60, temporary_directory=>\"/tmp/logstash\", s3_endpoint=>\"s3.amazonaws.com\", polling_frequency=>5>\n Error: undefined method `each' for #Aws::SQS::Types::Message:0x7f57c049", :level=>:error}