dinesh-rdk
commented
4 years ago And this is my initializer file config/initializers/shoryuken.rb
# Create SQS queues
require 'aws-sdk-sqs'
require 'shoryuken'
# We inline the jobs in development & testing
# Start mock server by "moto_server sqs -p 4576 -H localhost"
if Rails.env.development? || Rails.env.test?
client = Aws::SQS::Client.new(
region: ENV["AWS_REGION"],
access_key_id: ENV["AWS_ACCESS_KEY_ID"],
secret_access_key: ENV["AWS_SECRET_ACCESS_KEY"],
endpoint: 'http://localhost:4576',
verify_checksums: false
)
Shoryuken.configure_client do |config|
config.sqs_client = client
end
Shoryuken.configure_server do |config|
config.sqs_client = client
end
else
client = Aws::SQS::Client.new(
region: ENV['AWS_REGION'],
access_key_id: ENV['AWS_ACCESS_KEY_ID'],
secret_access_key: ENV['AWS_SECRET_ACCESS_KEY']
)
Shoryuken.sqs_client = client
end
# We have environment prepended queues in ActiveJob
all_queues = <AllQueues[]>
available_queues = client.list_queues.queue_urls.map{ |q| q.split('/').last }
queues_to_create = all_queues - available_queues
queues_to_create.each do |queue|
begin
puts "Creating queue #{queue}"
client.create_queue(
queue_name: queue,
attributes: {
"VisibilityTimeout" => "3600", # 3600 => 60 minutes
}
)
rescue => exception
puts "Error in creating queue #{queue}"
end
end
# After creating queues, make sure to subscribe with shoryuken
# RAILS_CACHE_CLASSES=true shoryuken -R -C ./shoryuken.yml
# By default, Shoryuken will make requests against SQS for getting the updated queue visibility timeout, so if you change the visibility timeout in SQS, Shoryuken will automatically update it.
# If you want to reduce the number of requests against SQS (therefore reducing your quota usage), you can disable this behavior by:
Shoryuken.cache_visibility_timeout = true
# Active Job allows you to prefix the queue names of all jobs. Shoryuken supports this behavior natively. By default, though, queue names defined in the config file (or passed to the CLI), are not prefixed in the same way. To have Shoryuken honor Active Job prefixes you must enable that option explicitly
Shoryuken.active_job_queue_name_prefixing = true
phstc
Hi @phstc Thank you for this wonderful library. I have been using RabbitMQ for queuing messages and now I am thinking of shifting to SQS.
I tested in my local with mock server as suggested in https://github.com/phstc/shoryuken/wiki/Using-a-local-mock-SQS-server
I have a lot (67) queues & I want them all to be independent (i.e. Don't want any queue to wait for any other queue to process)
So I configured shoryuken as
By default
delayis 0. So I expect shoryuken to poll messages from each queue in SQS every second.But the logs in mock server shows ~20 requests per second on average.
Because of it SQS seems very slow compared to Rabbit. Can you please guide me to make here?