search

OCA / queue

Asynchronous Job Queue
GNU Affero General Public License v3.0
183 stars 460 forks source link

[Question] queue_job when multiple odoo servers are used with load balancing (and single postgres) #422

Open JordiBForgeFlow opened 2 years ago

JordiBForgeFlow commented 2 years ago

In the scenario where multiple odoo servers are used with load balancing, and a single postgres server is used, will all of the odoo servers try to run all the jobs?

sbidoul commented 2 years ago

@JordiBForgeFlow you need to make sure that only one Odoo instance has queue_job in --load / server_wide_modules. This is very important.

Alternatively, you can launch a separate job runner process with something like #409.

sbidoul commented 2 years ago

See also https://github.com/OCA/queue/pull/256

github-actions[bot] commented 1 year ago

There hasn't been any activity on this issue in the past 6 months, so it has been marked as stale and it will be closed automatically if no further activity occurs in the next 30 days. If you want this issue to never become stale, please ask a PSC member to apply the "no stale" label.

daemonblade commented 1 year ago

@sbidoul having to tweak the configuration for one instance can become an issue when we have to deploy to a variable number of Odoo instances. Is it not possible to namescope the NOTIFY payload with host/main-odoo-pid/dbname to allow multiple job-runners to be active?

sbidoul commented 1 year ago

@sbidoul having to tweak the configuration for one instance can become an issue when we have to deploy to a variable number of Odoo instances. Is it not possible to namescope the NOTIFY payload with host/main-odoo-pid/dbname to allow multiple job-runners to be active?

@daemonblade in such situations, I don't add queue_job to server_wide_modules, and instead start the job runner independently, so all Odoo instances can share the same config.

Multiple job runners is tricky because it is currently all based on the queue being in memory.

b1zzu commented 1 year ago

Hi @sbidoul how can I run the queue_job independently? I've tried this PYTHONPATH=/opt/odoo/ python3 -m queue_job.jobrunner but doesn't work, it fails with this error:

AssertionError: Invalid import of queue_job.models.base.Base, it should start with 'odoo.addons'.
sbidoul commented 1 year ago

Try python3 -m odoo.addons.queue_job.jobrunner.

You'll need to add {queue-job-repo-dir}/setup/queue_job to PYTHONPATH or install odoo-addon-queue_job with pip.

daemonblade commented 1 week ago

I've reviewed our logs for multi-node systems with queue_job. We have one instance configured with queue_job in server_wide_modules. What I have noticed is that the instances that do NOT have queue_job in server_wide_modules will never run any jobs at all. Is this intended?