search

thelovemsg / chatting_app

0 stars 0 forks source link

What message broker and DB should I use? And What db should I use? #20

Open thelovemsg opened 1 year ago

thelovemsg commented 1 year ago

Objective

What message broker and DB should I use? And What db should I use?

Detailed work content

  • I'll discuss title what I should use alone.
thelovemsg commented 1 year ago

What is Redis? What is the pros and cons of it?

What is Redis?

The open source, in-memory data store used by millions of developers as a database, cache, streaming engine, and message broker.

Advantages:

  1. Speed: Redis's in-memory nature makes it extremely fast, which is excellent for a real-time system like a message broker.

  2. Pub/Sub Model: Redis natively supports a publish/subscribe model, making it easy to use as a message broker.

  3. Simple and Lightweight: Redis is relatively simple to set up and manage compared to more complex message brokers.

Considerations:

  1. Persistence: By default, Redis data, including messages, is not persisted to disk. If the Redis server goes down, in-flight messages can be lost. However, Redis does offer configurable persistence options to mitigate this.

  2. Delivery Guarantees: Redis does not have native support for complex delivery guarantees like "Exactly Once" or "At Least Once". Once a message is delivered to a subscriber, Redis has no knowledge of it. If a consumer fails to process a message, it could be lost.

=> In my case, I want to resend message if messages are not sent to users. So, Redis might be not a good choice.

  1. Scaling: While Redis is incredibly fast and can handle a lot of load for a single instance, scaling out can be more complex. It does support partitioning of data, but there's no native support for distributing pub/sub messages across a cluster. Each message is sent to all nodes in a cluster, which could be overkill if you have a large number of instances.
thelovemsg commented 1 year ago

What db should I use?

Some reasons why you might consider MongoDB or another NoSQL solution for a chat application:

  1. Schema-less: MongoDB is a NoSQL database that provides a flexible, JSON-like documents model which is more intuitive and can be changed over time. If your data requirements are constantly evolving, MongoDB could be a good fit.

  2. Scalability: MongoDB supports horizontal scaling through sharding, which could be important if your chat app grows rapidly. It's easier to handle large volumes of rapidly changing, structured, semi-structured, and unstructured data.

  3. Performance: MongoDB provides high performance for both reads and writes due to its support for in-memory processing, and it handles large volumes of rapidly changing data well.

  4. Real-time: MongoDB supports real-time integration of data better than many relational databases. This can be important in a chat app where messages must be delivered immediately.

There are also reasons to consider a relational database:

  1. Transactions: If your application requires complex transactions with multiple operations, relational databases are designed with this in mind, including support for rollback.

  2. Consistency: Relational databases are better at enforcing data integrity and consistency due to their ACID (Atomicity, Consistency, Isolation, Durability) properties.

  3. Query Capability: Relational databases have a powerful query language (SQL) that can handle complex queries. If your application needs complex joins, relational databases might be a better choice.

  4. Mature Technologies: Relational databases have been around for a long time and have proven solutions for a wide range of problems. There are a lot of tools and resources available for working with these databases.

So Good Reason To Use NoSQL DB For Chatting App.

  1. Easy horizontal scaling: Indeed, key-value stores like Cassandra, DynamoDB, or Riak are designed to be horizontally scalable. They can distribute data across many machines, which is beneficial for supporting large-scale applications. This feature is particularly important for chat applications that might need to grow rapidly to support increasing numbers of users and messages.

  2. Low latency: Key-value stores typically offer very low latency for both read and write operations, which is critical for chat applications that require real-time or near-real-time performance.

  3. Handling long tail of data: You're correct that relational databases can struggle with very large indexes and that key-value stores can handle this more efficiently. Random access in a large index can be expensive in terms of time and resources, and key-value stores handle this situation better.

  4. Proven use in reliable chat applications: The fact that large-scale, successful chat applications like Facebook Messenger and Discord use key-value stores is a strong argument in favor of using this type of database for a chat application. It shows that key-value stores can meet the requirements of these types of applications at scale.

thelovemsg commented 1 year ago

The Reason Of Low Latency Of NoSQL

  1. Simplified Data Models: Unlike relational databases, NoSQL databases often use simpler data models, such as key-value pairs, which allow for quicker access.

  2. Sharding: NoSQL databases often natively support horizontal scaling or sharding, which involves splitting the data across multiple servers. This means that as data grows, more servers can be added to keep read and write times low.

  3. Replication: NoSQL databases typically support data replication, storing copies of data on multiple nodes to ensure high availability and fast data access. Some NoSQL systems also support eventual consistency models, which can provide faster write times at the cost of potentially slightly delayed consistency across nodes.

  4. In-Memory Storage: Some NoSQL databases, like Redis, store data in memory as opposed to disk storage, which allows for very quick data access times. However, this can limit the total amount of data that can be stored and can increase costs.

  5. Denormalization and Aggregations: NoSQL databases often encourage denormalization, or the storing of related data together in a single document or value. This can eliminate the need for expensive join operations that are common in relational databases and result in faster query times.

  6. Indexing: Just like SQL databases, NoSQL databases also use indexing to speed up data retrieval. Depending on the NoSQL database type, various indexing techniques can be applied to reduce data access latency.

thelovemsg commented 1 year ago

What message broker is better for chatting app?

  1. RabbitMQ: This is a good general-purpose message broker that supports a variety of messaging patterns with a focus on reliability and delivery guarantees. It can handle high-throughput scenarios and offers features like message durability, priority queues, and more. However, it might be more complex to set up and administer compared to some other options.

  2. Kafka: Apache Kafka excels at handling streaming data and can handle extremely high volumes of messages. It's built to ensure that messages are stored and processed in a fast, highly available, and fault-tolerant manner. Kafka could be a good choice if you expect a very high volume of messages and need strong durability guarantees. However, Kafka can be complex to set up and manage, and might be overkill for smaller applications.

  3. Redis: As an in-memory data store, Redis is extremely fast and can be used as a simple message broker using its pub/sub capabilities. This could be a good choice if you need very low latency and don't need to store a large backlog of messages. However, Redis' pub/sub doesn't offer message persistence or delivery guarantees out of the box, so it might not be the best choice if you need those features.

  4. ActiveMQ: ActiveMQ is another robust option that offers a full set of features for messaging. It provides several advanced features such as support for JMS, message persistence, and transaction management. If your application is Java-based, ActiveMQ can be a suitable choice due to its strong integration with Spring and other Java frameworks.

thelovemsg commented 1 year ago

What are the defferences between message queue and message broker?

A message queue facilitates communication between applications by sending, receiving, and storing messages using the queue data structure. Even though we use message queues for data transfer, they can’t read the information they are carrying.

A message broker is simply a mechanism that extends the usage of message queues. Unlike message queues, message brokers can read the contents of the information carried through them. Also, message brokers can process information from all sorts of sources. For example, information read from files or HTTP requests.

In short, the message broker is responsible for: The conversion between data transport protocols The transformation of message formats between services The routing of the communication between services The distribution of events from various sources

As it turns out, message brokers use message queues to transmit information between all interested parties. Simply put, the message queue is a structure that stores the produced data until its consumption, and the message broker is a software component that manages the message queues.

reference: https://dreamix.eu/blog/dreamix/message-queue-vs-message-broker-whats-the-difference