tembo-io / pgmq

A lightweight message queue. Like AWS SQS and RSMQ but on Postgres.
PostgreSQL License
2.55k stars 58 forks source link

Consumer Retry Mechanism ? #274

Open jason810496 opened 2 months ago

jason810496 commented 2 months ago

Hi,

I am currently using pgmq as an alternative to RabbitMQ. I'm wondering if the current project supports a retry mechanism (like x-retry-count in RabbitMQ), or if I need to create another table to record the retry count to implement it.

I have read the documentation about Set the visibility timeout (VT) for a specific message, but I prefer a scenario where the message is archived after being retried X times.

Thank you for your help!

ChuckHend commented 2 months ago

Hi @jason810496, thanks for asking!

Each message has a field read_ct that increments every time the message is read with pgmg.read()

select * from pgmq.read(queue_name => 'myqueue', vt => 10, qty => 1);
 msg_id | read_ct |          enqueued_at          |              vt               |      message       
--------+---------+-------------------------------+-------------------------------+--------------------
      1 |       1 | 2024-07-01 12:39:02.198732+00 | 2024-07-01 12:39:28.176894+00 | {"hello": "world"}

There isn't a mechanism that will automatically move the message to another queue (like a DLQ or something). But what some projects do is have their consumer check the read_ct value then move/archive/delete the message if it exceeds some value.

v0idpwn commented 2 months ago

Another project using the read count to archive upon processing failure

jason810496 commented 2 months ago

Thanks for the reply, @ChuckHend 🙏 I will use read_ct to mark the message as failed to process.

jason810496 commented 2 months ago

Hi, @v0idpwn Do you have any description on how to use the project? Thanks.

v0idpwn commented 2 months ago

@jason810496 there's some info in the documentation and on broadway documentation