Struggling to get LiteJob/LiteQueue workers to run

[colinframe]

I've got a pretty simple Rails 7.1 / Ruby 3.2.2 app on litestack 0.4.3.

I can see that jobs are being enqueued for both ActiveJob jobs and ActionMailer emails queued with deliver_later.

The app is configured with config.active_job.queue_adapter = :litejob and config.active_job.queue_name_prefix = Rails.env.

I've got a very simple litejob.yml which configures a few queues, taken from the docs, and the config displayed in Litejobqueue.jobqueue looks correct, as best as I can tell;

irb(main):003> Litejobqueue.jobqueue
=> 
#<Litejobqueue:0x00007f012f4e5a98
 @conn=
  #<Litesupport::Pool:0x00007f012f505320
   @block=
    #<Proc:0x00007f012f4e52a0 /usr/local/bundle/ruby/3.2.0/gems/litestack-0.4.3/lib/litestack/litesupport.rb:235>,
   @count=1,
   @mutex=#<Litesupport::Mutex:0x00007f012f4e5188 @mutex=#<Thread::Mutex:0x00007f012f4e5110>>,
   @resources=#<Thread::Queue:0x00007f012f4e5228>>,
 @gc=
  #<Thread:0x00007f012f4e16a0 /usr/local/bundle/ruby/3.2.0/gems/litestack-0.4.3/lib/litestack/litejobqueue.rb:214 sleep>,
 @jobs_in_flight=0,
 @logger=
  #<Logger:0x00007f012f4e1fb0
   @default_formatter=#<Logger::Formatter:0x00007f012f4e1e70 @datetime_format=nil>,
   @formatter=nil,
   @level=0,
   @logdev=nil,
   @progname=nil>,
 @mutex=#<Litesupport::Mutex:0x00007f012f4e1588 @mutex=#<Thread::Mutex:0x00007f012f4e1560>>,
 @options=
  {:config_path=>"./config/litejob.yml",
   :path=>#<Pathname:./storage/production/queue.sqlite3>,
   :queues=>[["default", 1], ["urgent", 5], ["critical", 10]],
   :workers=>5,
   :retries=>5,
   :retry_delay=>60,
   :retry_delay_multiplier=>10,
   :dead_job_retention=>864000,
   :gc_sleep_interval=>7200,
   :logger=>nil,
   :sleep_intervals=>[0.001, 0.005, 0.025, 0.125, 0.625, 1.0, 2.0],
   :metrics=>false},
 @queues=[[10, [["critical", false]]], [5, [["urgent", false]]], [1, [["default", false]]]],
 @running=true,
 @workers=
  [#<Thread:0x00007f012f4e1bc8 /usr/local/bundle/ruby/3.2.0/gems/litestack-0.4.3/lib/litestack/litejobqueue.rb:182 sleep>,
   #<Thread:0x00007f012f4e1ab0 /usr/local/bundle/ruby/3.2.0/gems/litestack-0.4.3/lib/litestack/litejobqueue.rb:182 sleep>,
   #<Thread:0x00007f012f4e19c0 /usr/local/bundle/ruby/3.2.0/gems/litestack-0.4.3/lib/litestack/litejobqueue.rb:182 sleep>,
   #<Thread:0x00007f012f4e18d0 /usr/local/bundle/ruby/3.2.0/gems/litestack-0.4.3/lib/litestack/litejobqueue.rb:182 sleep>,
   #<Thread:0x00007f012f4e17e0 /usr/local/bundle/ruby/3.2.0/gems/litestack-0.4.3/lib/litestack/litejobqueue.rb:182 sleep>]>

That looks to me like the workers are setup but none of the jobs in the queues are being worked off at all.

Is there something super obvious I'm missing?

Update: I forgot to mention that this is also the case in development mode as well; I can successfully enqueue jobs but they're never picked up by the workers.

[oldmoe]

I am suspecting it could be the name prefix, did you ever try without it?

[oldmoe]

Ok, they definitely will not run unless you have the queue names in litejob.yml with the name prefix as well,

i.e. development_critical, etc.

[oldmoe]

Also, there is not need for prefixing with the environment anyway, Litestack will create a separate sqlite database for each environment by default, there is no need for further name spacing

[clayton]

EDIT: I found a resolution

I had an old docker container running from a previous version of the app. I think it was the result of switching from using kamal deploy to the more specific kamal deploy -d production with some production specific environment variables etc. etc.

Anyway, not sure why that had an effect on the queue or workers, but maybe the old workers running in the stale container were picking up the jobs but failing (and endlessly retrying?)

[Original issue below]

I'm having the same issue.

Rails v7.1.3.2 litestack v0.4.3

Works fine in development, but not on my kamal deployed VPS.

I am specifying litejob in application.rb

config.active_job.queue_adapter = :litejob

I only have one queue default and every job goes into that queue.

I see the jobs enqueued in the logs but they are never picked up by the workers.

irb(main):001> Litejobqueue.jobqueue
=> 
#<Litejobqueue:0x00007f298d535170
 @conn=
  #<Litesupport::Pool:0x00007f298d5533f0
   @block=
    #<Proc:0x00007f298d534c20 /usr/local/bundle/ruby/3.3.0/gems/litestack-0.4.3/lib/litestack/litesu
pport.rb:235>,
   @count=1,
   @mutex=#<Litesupport::Mutex:0x00007f298d534b08 @mutex=#<Thread::Mutex:0x0000
7f298d534a90>>,
   @resources=#<Thread::Queue:0x00007f298d534ba8>>,
 @gc=
  #<Thread:0x00007f298de8f4f0 /usr/local/bundle/ruby/3.3.0/gems/litestack-0.4.3/lib/litestack/litejo
bqueue.rb:214 sleep>,
 @jobs_in_flight=0,
 @logger=
  #<Logger:0x00007f298d553080
   @default_formatter=#<Logger::Formatter:0x00007f298d531b10 @datetime_format=
nil>,
   @formatter=nil,
   @level=0,
   @level_override={},
   @logdev=nil,
   @progname=nil>,
 @mutex=#<Litesupport::Mutex:0x00007f298de8f1a8 @mutex=#<Thread::Mutex:0x00007f
298de8f108>>,
 @options=
  {:config_path=>"./config/litejob.yml",
   :path=>#<Pathname:/data/production/queue.sqlite3>,
   :queues=>[["default", 1]],
   :workers=>5,
   :retries=>5,
   :retry_delay=>60,
   :retry_delay_multiplier=>10,
   :dead_job_retention=>864000,
   :gc_sleep_interval=>7200,
   :logger=>nil,
   :sleep_intervals=>[0.001, 0.005, 0.025, 0.125, 
0.625, 1.0, 2.0],
   :metrics=>false},
 @queues=[[1, [["default", false]]]],
 @running=true,
 @workers=
  [#<Thread:0x00007f298d5318e0 /usr/local/bundle/ruby/3.3.0/gems/litestack-0.4.3/lib/litestack/litej
obqueue.rb:182 sleep>,
   #<Thread:0x00007f298d5317f0 /usr/local/bundle/ruby/3.3.0/gems/litestack-0.4.3/lib/litestack/litej
obqueue.rb:182 sleep>,
   #<Thread:0x00007f298d531700 /usr/local/bundle/ruby/3.3.0/gems/litestack-0.4.3/lib/litestack/litej
obqueue.rb:182 sleep>,
   #<Thread:0x00007f298d531610 /usr/local/bundle/ruby/3.3.0/gems/litestack-0.4.3/lib/litestack/litej
obqueue.rb:182 sleep>,
   #<Thread:0x00007f298d531520 /usr/local/bundle/ruby/3.3.0/gems/litestack-0.4.3/lib/litestack/litej
obqueue.rb:182 sleep>]>