Lack of clean up of old queues ( orphaned queues)

[kylelikesboatstuff]

Seems like the way the queues are maintained, it's not set up to garbage collect after a deployment. Leaving me with:

Seems like everyone would have this issue. What did I mess up? Can I set them to just use the same persistent queue?

[andrcuns]

You can add following config to clean up queues. Currently it's not done automatically:

config.shutdown_callback = proc do
  Sidekiq::Queue.all.find { |q| q.name == "#{config.queue_prefix}-#{SidekiqAlive.hostname}" }&.clear
end

[kylelikesboatstuff]

We're running on version 2.1.7 (because we are still on ruby 2.5.9), and the shutdown_callback isn't available in that version. Version 2.1.7 does cleanup after itself, but we found our shutdown process wasn't being run. In our k8s file, we were calling a bash script starting our sidekiq process with:

bundle exec sidekiq -c 10 -e $APP_ENV -q default -q mailers

When k8s sends a shutdown command bash catches, it doesn't get propagated to sidekiq, so we were just rug pulling without allowing shutdown callbacks to run on the eventual termination grace period expiration. To pass the signals to the sidekiq process, we had to run the command with the bash exec call. This, instead of forking and effectively creating a do-nothing middleware, promotes the would-be child to the parent. As such the passthrough of signals ( like SIGINT) from k8s now actually hit sidekiq and it can act appropriately.

exec bundle exec sidekiq -c 10 -e $APP_ENV -q default -q mailers

[gdubicki]

Can you please consider making cleaning old queues the default setting, @kylelikesboatstuff & @arturictus ? I already created a PR for it ☝️. I can't really imagine the use cases where you don't want to autoclean them... Their existence is breaking the default queue latency for me.