search

raystack / raccoon

Raccoon is a high-throughput, low-latency service to collect events in real-time from your web, mobile apps, and services using multiple network protocols.
https://raystack.github.io/raccoon/
Apache License 2.0
199 stars 29 forks source link

Disk-backed Persistent queue for channels in Raccoon #52

Open stym06 opened 2 years ago

stym06 commented 2 years ago

Summary Currently, Raccoon uses Channels for the intermediate processing of EventRequests, which are then forwarded to the message broker. But it does not solve the problem of loss of events that are there in the channel and could not get forwarded to Kafka when the server dies.

Proposed solution Implement some disk-backed queueing for intermediate persistence, similar to this project. https://github.com/jhunters/bigqueue

AkbaraliShaikh commented 2 years ago

we are already working on the event guarantee delivery proposal, will share the proposal soon, thanks.

stym06 commented 2 years ago

Is there a running Design Doc for this? Would love to see the progress and the approach.

stym06 commented 1 year ago

Guys, is there any progress on this? Is there a design doc that I can look into?