search

ruby-shoryuken / shoryuken

A super efficient Amazon SQS thread based message processor for Ruby. This project is in MAINTENANCE MODE. Reach me out on Slack link on the description if you want to become a new maintainer.
Other
2.06k stars 280 forks source link

Exception in Shoryuken::Queue#receive_messages #752

Closed mkeemon closed 1 year ago

mkeemon commented 1 year ago

Last night, we saw our Shoryuken processes begin to fail with the below error. It appears that the aws-ruby-sdk SQS client's receive_message function is sometimes returning an empty array and sometimes returning nil. It's unclear if this is a change in the aws-sdk gem behavior or the AWS APIs. My guess would be the latter since this happened overnight in between releases to our application.

Shoryuken -- Manager failed: undefined method `map' for nil:NilClass
.../shoryuken-6.1.0/lib/shoryuken/queue.rb:46:in `receive_messages'

We've monkey patched this in our application for the time being:

module Shoryuken
  class Queue
    def receive_messages(options)
      messages = client.receive_message(options.merge(queue_url: url)).messages || []
      messages.map { |m| Message.new(client, self, m) }
    end
  end
end
phstc commented 1 year ago

Hi @mkeemon

That's unexpected. I would guess something on the AWS side.

What's your aws-sdk-sqs version? Did it change recently?

Would you be able to submit a PR with the failsafe change?

jsteinberg commented 1 year ago

We saw the same issue also start last night. There is some explanation in this issue here:

https://github.com/aws/aws-sdk-ruby/issues/2947

mkeemon commented 1 year ago

All set @phstc https://github.com/ruby-shoryuken/shoryuken/pull/753. Let me know if you need anything else or if I did something improperly!

Edit: We're using 1.67.0 of the aws-sdk-sqs gem in multiple projects, all of which began failing last night