search

timgit / pg-boss

Queueing jobs in Postgres from Node.js like a boss
MIT License
2.04k stars 157 forks source link

Expiring a job that didnt start running? #205

Closed noamgat closed 3 years ago

noamgat commented 3 years ago

Hi,

We use pg-boss in a use case, where some of the jobs are fulfilled by external workers. We added an API call to our server that fetches a job in a queue, and the workers poll that API call. We would like to time out jobs in a way, that if workers are down, jobs would time out. However, the expiry options are only about jobs that started running - if the job crashed in the middle, it will expire and retry. It does not cover the case where the process polling for the job crashes altogether.

Am I missing something? Is there a way to know when this happens?

timgit commented 3 years ago

Yes, when you call fetch() to pull a job from the queue, it's immediately set into active state. pg-boss monitors for this and will expire the job if you don't call complete() or fail() within the expiration interval.

noamgat commented 3 years ago

Calling fetch() is done by the worker. What if the worker is down?

On Thu, Nov 5, 2020 at 4:09 AM Tim Jones notifications@github.com wrote:

Yes, when you call fetch() to pull a job from the queue, it's immediately set into active state. pg-boss monitors for this and will expire the job if you don't call complete() or fail() within the expiration interval.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/timgit/pg-boss/issues/205#issuecomment-722079640, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAKFA2EJZQ4PGN43EE55R7TSOICM7ANCNFSM4TJI65HQ .

timgit commented 3 years ago

If the worker is down, the jobs will remain in the queue in created state.