search

janbjorge / pgqueuer

PgQueuer is a Python library leveraging PostgreSQL for efficient job queuing.
https://pgqueuer.readthedocs.io/en/latest/index.html
MIT License
969 stars 13 forks source link

[feature request] Named pipes #18

Closed tobwen closed 4 months ago

tobwen commented 5 months ago

Feature request

I don't know if the priority is just moving a job up or if it is also a type of “named pipe” as used in BeanstalkD. With BeanstalkD, for example, you can have a pipe “mail” for sending mails and a pipe "rendering" to do some rendering jobs.

You could of course implement this at payload level, but then it won't appear in the dashboard.

janbjorge commented 5 months ago

Priority just moves the job, i sort by priority, then id. ref. https://github.com/janbjorge/PgQueuer/blob/595afe229fa99daa30e06fc4edbc15b88e9d6c8d/src/PgQueuer/queries.py#L203 its inspired by python PriorityQueue but i pick the items with the highest priority first.

You can create new in a entrypoint-funciton but i would not recommend that is i think you quickly will get "positive feedback loops".

Ive never used or heard about BeanstalkD, so i cant comment on named pipes part.

janbjorge commented 4 months ago

Thank you for your feature request and the explanation. Your suggestion about implementing a feature similar to BeanstalkD's "named pipes" for different types of jobs is insightful. However, as of now, I don't plan to build this feature into PgQueuer. I appreciate your understanding and encourage you to continue sharing any other ideas or feedback you may have.