mgcrea / prisma-queue

Minimalist postgresql job queue for Prisma
MIT License
47 stars 4 forks source link

Q: How to best organize queues? #1

Closed juni0r closed 4 months ago

juni0r commented 8 months ago

Hi,

I'm using prisma-queue in my current project to great delight. Thanks for sharing this! 🚀

I was wondering how to best organize my queues. Since the handler is typed to a certain type of Job, it would seem that I have to either create a separate queue for each job type or create a generic job type that handles different tasks.

Say, for instance, you want to have a priority queue for sending emails and another one for all other jobs. How would you set this up?

mgcrea commented 4 months ago

Sorry for the late comment, I think both options are possible and valid and ends up being a trade-off between simplicity and flexibility.

One single queue with a job type field/enum should work for most cases, but you have a single configuration and share the concurrency between all job types.

Multiple queues let you more cleanly separate job types and have flexible per-queue configuration, but you now have multiple queues running concurrently that you might want to move to their own processes to guarantee isolation which can lead to a more complex setup.