RFC: Implementing CQRS Architecture with Scalable Search Functionality and Future Upgrades

[vinit717]

Summary

This RFC outlines the strategy for enhancing our project's functionality using a Command Query Responsibility Segregation (CQRS) architecture. The primary focus is to optimize search performance through Elasticsearch, Azure Cosmos DB (Free Tier), or a traditional database, while also ensuring future scalability, cost-effectiveness, and the flexibility to accommodate new features.

Motivation

The project's immediate objective is to upgrade the search functionality while keeping future enhancements in mind. By adopting a CQRS architecture, we aim to improve the performance and maintainability of the application, ensuring a seamless user experience and flexibility for future feature additions.

Proposed Solution

1. Write Side (Command):

   • Retain Firestore as the primary database for write operations, including tasks' creation, updates, and deletions.

2. Read Side (Query):

   • Analyze potential read-optimized databases for the search functionality within the CQRS architecture.
   • Implement synchronization mechanisms to ensure consistent data between the write and read sides.

3. Search Functionality:

   • Develop frontend search functionality that queries the read-optimized database, enhancing performance and user experience.

4. Future Upgrades:

   • Design the architecture to allow seamless incorporation of additional features beyond search, ensuring maintainability and scalability.

Database Options and Analysis

1. Elasticsearch:

   • Pros: Advanced full-text search, scalability, relevance scoring, complex queries, vibrant community support.
   • Cons: Learning curve, operational complexity, potential costs (consider open-source alternatives).

2. Azure Cosmos DB (Free Tier):

   • Pros: Cost-effectiveness, scalability, multi-API support, global distribution, seamless Azure ecosystem integration.
   • Cons: Learning curve, synchronization design, performance limitations (evaluate free tier capabilities).

3. Traditional Database:

   • Pros: Familiarity, potential cost savings, architectural control, tailored optimization for specific use cases.
   • Cons: Limited full-text search capabilities, potential performance challenges for complex queries.

Cost and Performance Considerations

• Cost: Evaluate the cost factors associated with each database option, accounting for operational expenses, scalability, and potential open-source alternatives.
• Performance: Perform thorough performance testing for each option to assess string searching, complex queries, and scalability under varying load levels.

Implementation Strategy

1. Database Selection:

   • Evaluate the pros and cons of each database option based on project requirements, cost, performance, and operational feasibility.

2. Synchronization Process:

   • Design a synchronization process to keep the chosen read-optimized database up-to-date with Firestore's write operations. Leverage Firestore triggers, cloud functions, or a custom backend service for efficient synchronization.

3. Future-Proof Architecture:

   • Design the architecture to accommodate future feature upgrades while maintaining performance and flexibility.

Conclusion

Selecting the most suitable read-optimized database is crucial for achieving our project's goals of implementing CQRS and enhancing search functionality. Considering the pros, cons, cost, and performance factors of Elasticsearch, Azure Cosmos DB (Free Tier), and traditional databases will guide our decision.

Please comment with your valuable feedback to achieve this in our project

[askprateek]

You should check out https://typesense.org/ It's an open-source alternative to Algolia.

It offers the best interface for implementing search. I've used Algolia, and Typesence has a similar API and uses the same SDK. High recommended.

[vikhyat187]

Hey @vinit717 @sahsisunny can we also mention the problem exactly being solved in the ticket.