How do we separate queues on different servers?

rails / solid_queue

Database-backed Active Job backend

MIT License

1.95k stars 131 forks source link

How do we separate queues on different servers? #413

Closed ratnarajsukale closed 1 week ago

ratnarajsukale commented 1 week ago

I have two queues:

default
live_updates

I also have two VPS servers. I want to run these queues on different servers because their resource and priority requirements are different.

The only command I found online is:

bundle exec rake solid_queue:start

Which i think will take any ready to execute job & not from any specific queue. I tried giving --queue flag but the container won't even start saying this flag won't work. Then i tried -q flag, this will start the worker but takes jobs randomly. Also is there any documentation for it that i missed?

rosa commented 1 week ago

Hey! You can use different config files and specify which queues you want in each server.

bin/jobs --help
Commands:
  jobs help [COMMAND]  # Describe available commands or one specific command
  jobs start           # Starts Solid Queue supervisor to dispatch and perform enqueued jobs. Default command.

Options:
  -c, [--config-file=SOLID_QUEUE_CONFIG]                          # Path to config file (default: config/queue.yml).
      [--recurring-schedule-file=SOLID_QUEUE_RECURRING_SCHEDULE]  # Path to recurring schedule definition (default: config/recurring.yml).
      [--skip-recurring]                                          # Whether to skip recurring tasks scheduling
                                                                  # Default: false

ratnarajsukale commented 1 week ago

Hi, Thanks for the help, just realised i do not have bin/jobs. Then found out i was using gem "solid_queue", "~> 0.3.1" LOL!