arturictus / sidekiq_alive

Liveness probe for Sidekiq in Kubernetes deployments
MIT License
189 stars 57 forks source link

queue_prefix config not used #42

Closed c2ofh closed 1 year ago

c2ofh commented 4 years ago

I wanted to use the queue_prefix for the queues, but the name is still "sidekiq_alive-:hostname"

init:

SidekiqAlive.setup do |config|
  config.queue_prefix = :x_alive
  config.server = 'puma'
end

I want to move this queues in all displays to the end, cause they normally sorted by name.

image

Startouf commented 2 years ago

Do you have a workaround ? working patch ?

c2ofh commented 2 years ago

@Startouf no the name is still default. I just live with this bug.

Startouf commented 2 years ago

@c2ofh I've been looking at the code and I believe this may be a problem related to the startup sequence. Where did you put your init code ? Is it, by any chance, in an initializer that would run before/after a sidekiq initializer ?

@arturictus where do you put your sidekiq alive init code ?

I have an initializer that may interfere in my rails app.

# config/initializers/sidekiq.rb
Sidekiq.configure_server do |config|
  table_name_prefix = ::ActiveJob::Base.queue_name_prefix
  table_name_delimiter = ::ActiveJob::Base.queue_name_delimiter

  config.options[:queues] = config.options[:queues].map do |queue_name|
    [table_name_prefix, queue_name].compact.join(table_name_delimiter)
  end

  config.redis = {
    url: ENV["SIDEKIQ_REDIS_URL"],
    password: ENV["SIDEKIQ_REDIS_PASSWORD"]
  }
end
arturictus commented 2 years ago

Hi @Startouf, You can review the configurations here: https://github.com/arturictus/sidekiq_alive/blob/master/lib/sidekiq_alive.rb#L10-L36 I do not know what is the configuration result with the code above. Can you inspect in a worker Sidekiq.options and paste it here? I put my sidekiq_alive.rb in initializers folder. Thanks

Startouf commented 2 years ago

My sidekiq options look like this

irb(main):001:0> Sidekiq.options
=> 
{:queues=>[],
 :labels=>[],
 :concurrency=>10,
 :require=>".",
 :strict=>true,
 :environment=>nil,
 :timeout=>25,
 :poll_interval_average=>nil,
 :average_scheduled_poll_interval=>5,
 :error_handlers=>[#<Sidekiq::ExceptionHandler::Logger:0x000055da49937a00>],
 :death_handlers=>[],
 :lifecycle_events=>{:startup=>[], :quiet=>[], :shutdown=>[], :heartbeat=>[]},
 :dead_max_jobs=>10000,
 :dead_timeout_in_seconds=>15552000,
 :reloader=>#<Proc:0x000055da48f14500 /usr/local/bundle/gems/sidekiq-6.2.2/lib/sidekiq.rb:38>}

There should be something in the lifecycle hooks ?

I have

SidekiqAlive.setup do |config|
  config.server = 'puma'
  config.queue_prefix = "#{::ActiveJob::Base.queue_name_prefix}_z_sidekiq_alive"
end

Sidekiq alive does start with puma, so at least my settings are working partly

I am starting sidekiq in kubernetes using bundle exec sidekiq -C config/sidekiq.yml

The workers registered the good queue in sidekiq, when I go to the sidekiq UI, in busy I see my servers with Queues: MyApp_production_sidekiq_alive-hermes-sidekiq-64fd9f9fb9-95vsk, which correspond to my hostname + prefix (but as you can see not the prefix I added in sidekiq_alive). The problem is that the jobs are enqueued (at least the first job) on queues without the prefix sidekiq_alive-hermes-sidekiq-64fd9f9fb9-95vsk

EDIT

When I open a shell and I SidekiqAlive.current_queue, it returns something like => "MyApp_staging_z_sidekiq_alive-hermes-sidekiq-55fbf9bf8f-zdbvm" so I believe the config is only applied after SidekiqAlive has initialized and registered itself, which is causing the problem

Startouf commented 2 years ago

I was able to fix the problem by moving the SidekiqAlive::Worker.sidekiq_options queue: current_queue line into the sq_config.on(:startup) do block, I'm not sure this may affect existing installations or not, I have opened the above PR and it would be helpful if people with an existing sidekiq alive installation could test this new config to confirm whether it affects them or not