How to arrive at the latency for a given Shoryken::Queue?

[fluke]

Sidekiq implements a method Sidekiq::Queue.latency that calculates the queue's latency, the difference in seconds since the oldest job in the queue was enqueued. What's a good way to implement something similar for Shoryuken. Would also need to get the queue depth but I think that's possible to figure out from queue attributes ApproximateNumberOfMessages.

https://github.com/mperham/sidekiq/blob/48bad6e7ec758d718862ddee0e3811edd1eefb05/lib/sidekiq/api.rb#L265

[matt-taylor]

Hi @fluke AWS Cloudwatch has a metric called ApproximateAgeOfOldestMessage that will get you the age of the oldest message. https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-available-cloudwatch-metrics.html

Does that work for you or are you asking for Shoryuken to expose these metric explicitly?

[fluke]

That works. I am looking to get a list of all queues currently defined and get the latency and number of messages in order to set up autoscaling for my workers. What's the way to get all the queues that are set up?

Shoryuken::Queue could possibly expose the queue_attributes that would be helpful, right now it's a private method.

[matt-taylor]

What's the way to get all the queues that are set up?

Shoryuken provides a CLI to gather all Queues. https://github.com/ruby-shoryuken/shoryuken/wiki/Using-a-local-mock-SQS-server#running-sqs-commands-locally

You can check out all CLI commands by executing.

bundle exec shoryuken sqs ls

root@8af778501e5e:/gem# bundle exec shoryuken help sqs
Commands:
  shoryuken sqs create QUEUE-NAME                       # Create a queue
  shoryuken sqs delete QUEUE-NAME                       # delete a queue
  shoryuken sqs dump QUEUE-NAME                         # Dumps messages from...
  shoryuken sqs help [COMMAND]                          # Describe subcommand...
  shoryuken sqs ls [QUEUE-NAME-PREFIX]                  # Lists queues
  shoryuken sqs mv QUEUE-NAME-SOURCE QUEUE-NAME-TARGET  # Moves messages from...
  shoryuken sqs purge QUEUE-NAME                        # Deletes the message...
  shoryuken sqs requeue QUEUE-NAME PATH                 # Requeues messages f...

Options:
  -e, [--endpoint=ENDPOINT]  # Endpoint URL
                             # Default: http://localstack:4566

[fluke]

Thanks a lot. This has been helpful.