Making job check interval a function of job type

timgit / pg-boss

Queueing jobs in Postgres from Node.js like a boss

MIT License

2.13k stars 158 forks source link

Making job check interval a function of job type #5

Closed jtlapp closed 8 years ago

jtlapp commented 8 years ago

newJobCheckInterval is configured for all of pg-boss, but different job types can have different interval requirements. Consider the following job types:

send health check to cluster node (might be every 10 minutes)
download an mp3 from a web site (might be once per minute)
delete a file (might be once per 100ms -- might be queued because many files)

It would be nice to not only configure this as a function of job type, but to be able to dynamically change it as a function of job type, in order to support throttling (see #4).

jtlapp commented 8 years ago

But I don't really know how people use this polling interval in practice. You've also got a delay time, which is functionally different but still related. How is a single polling interval selected for many different job types? Seems like it would have to be chosen independently of the jobs performed.

jtlapp commented 8 years ago

There seems to be a workaround for this: create a different instance of pg-boss for each type of job.

timgit commented 8 years ago

newJobCheckInterval is an internal configuration setting that I'm being polite to expose for the sake of giving consumers more knobs and buttons to adjust. You won't likely need to change it.

timgit commented 8 years ago

you shouldn't need to create multiple pg-boss instances for that.

At this point, I think you'd be better off just throwing your application together using a single instance with multiple publish and subscribes of all of your job types.

jtlapp commented 8 years ago

Closing until I've made a better study of the code. Thanks for your help!

Globik commented 6 years ago

newJobCheckInterval 's default value is 1 second. But 1 day is also ok.