Real-time Jobs

[kydemy-fran]

Sorry to bother you again @vgarvardt

We were using an internal library before moving to gue. (We have been using both until recently we removed the old one) On that library we had a functionality that might be ported to gue and might be interesting.

Basically gue works polling jobs from the DB using an interval. If you have many jobs types you have to basically find a balance for setting that interval, that is not polling too much, but you do not add too much delay processing important jobs.

The idea would be to:

1. Mark jobs (when they get enqueued) as real-time.
2. Have a trigger in the DB that when a job gets inserted use the postgresql NOTIFY [channel] functionality.
3. GUE has a worker awaiting using the: LISTEN [channel] functionality.
4. When a new message is received, triggers the timer so it wakes up the pool associated to the job.Type. (or something like this. we would need to check if they are currently running as a fail-safe)
5. Workers will run as usual picking up that task.

PS: I've read your comment about the plugin system. But this would need to be CORE I believe to access the timer, workmap, etc.

Let me know what you think, thank you.

[vgarvardt]

I have bad experience with LISTEN/NOTIFY so it will not be implemented in this library, having only interval polling was the design decision I made at the beginning and I do not want to change it

[kydemy-fran]

Ok, no worries :+1: Thank you for the reply.