search

tracehubpm / tracehub

Tracehub and Project as a Code, a VCS-based collaboration tool
MIT License
5 stars 0 forks source link

Handle incoming messages using distributed queue #51

Closed h1alexbel closed 8 months ago

h1alexbel commented 8 months ago

We should not process messages tracehub bot receives, instead we should push message in the queue, and consumer of this message will process event later by polling it

0pdd commented 8 months ago

@h1alexbel the puzzle #128 is still not solved.

h1alexbel commented 8 months ago

this issue was decomposed into #128, lets use Rabbitmq for it, its a robust message broker, we probably gonna use its point-to-point delivery system. Other option would be Redis queuing.

h1alexbel commented 8 months ago

image

and this from AWS guide:

Message persistence RabbitMQ supports persistent messages and transient messages. When it sends messages to the persistent queue, it writes data to permanent storage as soon as it arrives. RabbitMQ also writes transient messages to the disk, but only if they exceed the memory’s capacity.

On the other hand, Redis does not support persistent messages by default. Developers must enable a feature called Redis Database (RDB) to take periodic snapshots of the RAM and store them on disk. Enabling data persistence on Redis adds overhead to data operations, which slows down message delivery. Another alternative is to use recovery techniques like asynchronous replication.

h1alexbel commented 8 months ago

lets stick to RabbitMQ