search

timgit / pg-boss

Queueing jobs in Postgres from Node.js like a boss
MIT License
2.13k stars 158 forks source link

deployment models #215

Closed basaran closed 3 years ago

basaran commented 3 years ago

Hello,

Thank you for your dedicated work on pg-boss and making it available to us. I installed it today and started looking into it. After an hour or so, I realized I was wrong with my expectations on logical flow of events. Please allow me to explain:

I was mostly looking into rabbitMq before I found out about pg-boss, and since rabbitMq is an application on its own, I was thinking pg-boss would do its magic as:

image

But then I realized, by default, everything works in a single container (which is also great), such as:

image

Could you please advise what would be the recommended method if we would like to move pg-boss away from the express application?

I found one thread where a user was reporting he disabled the monitoring:

https://github.com/timgit/pg-boss/issues/98

I assume he's referring to:

https://github.com/timgit/pg-boss/blob/master/docs/configuration.md#maintenance-options

But I could be wrong.

If you could please shed some light when you have time, I will surely appreciate it.

timgit commented 3 years ago

You can run as many instances of pg-boss wherever you want. The queue monitoring is multi-master compatible, so it won't hurt things to add instances to the same backing database. Each instance has to decide how to interact with the queues, so if you create an instance in express, for example, but never use boss.subscribe(), it's not going to be trying to fetch jobs.

basaran commented 3 years ago

thank you for getting back, so to create a decoupled system, is it safe to have

Express containers which will boss.publish

noSupervisor: true
noScheduling: true

Worker container which will boss.subscribe

noSupervisor: true
noScheduling: true

and pg-boss container which will do all the maintenance, and scheduling towards the postgres instance.

noSupervisor: false
noScheduling: false

What would happen if all three had the defaults? Would it cause any harm?

timgit commented 3 years ago

You don't have to worry about the supervisor and scheduling options in the constructor. "It Just Works" :)

basaran commented 3 years ago

Thank you :)

basaran commented 3 years ago

By the way, when I saw the snake_case application_name, in less than a second I already knew what application name it was. Thank you for these fine details.