anitsh
commented
1 year ago What are the questions am I asking? If it's a fork of NATS with improvement, what are those.
Open anitsh opened 1 year ago
anitsh
commented
1 year ago What are the questions am I asking? If it's a fork of NATS with improvement, what are those.
https://github.com/memphisdev/memphis-broker
Memphis{dev} is an open-source real-time data processing platform that provides end-to-end support for in-app streaming use cases using Memphis distributed message broker.
Memphis{dev} enables building next-generation applications that require large volumes of streamed and enriched data, modern protocols, zero ops, rapid development, extreme cost reduction, and a significantly lower amount of dev time for data-oriented developers and data engineers.
Broker - Messaging Queue. Memphis broker is a fork of NATS.io, which is an existing and battle-tested messaging queue with Memphis improvements and tunings. https://www.youtube.com/watch?v=hjXIUPZ7ArM&t
MongoDB - Only for UI state persistency (not used for storing messages).
Consumers are pull-based. The pull interval and the batch size can be configured. Each consumer will consume all the messages residing inside a station. The user must create consumers within the same consumer group if a client requires a horizontal scale and split messages across different consitency group members.
RAFT is used as the consensus algorithm responsible for atomicity within Memphis.
Memphis forked and modified NATS as its core queue.
The NATS streaming protocol sits atop the core NATS prohttps://github.com/obumnwabude/fast-chat/tree/articletocol and uses Google's Protocol Buffers. Protocol buffer messages are marshaled into bytes and published as Memphis messages on the specific station.
In-App Data Streaming Platform
Implementation Examples: https://github.com/obumnwabude/fast-chat/tree/article https://github.com/yanivbh1/FastMart