anitsh
commented
3 years ago Event-Driven
The microservice architecture style is for creating (backend) services that are typically activated using one of these three general types of connectors: HTTP call (to a REST service) RPC-like call using a platform-specific component technology, such as gRPC or GraphQL An asynchronous message that goes through a queue in a message broker
The first two are typically synchronous, HTTP calls being the most common alternative. Often, services need to call others forming a service composition, and many times the interaction in a service composition is synchronous. If instead, we create (or adapt) the participating services to connect and receive messages from a queue/topic, we’ll be creating an event-driven architecture. (One can debate the difference between message-driven and event-driven, but we’ll use the terms interchangeably to represent asynchronous communication over the network using a queue/topic provided by a message broker product, such as Apache Kafka, RabbitMQ, and Amazon SNS.) An important benefit of an event-driven architecture is improved scalability and throughput. This benefit stems from the fact that message senders are not blocked waiting for a response, and the same message/event can be consumed in parallel by multiple receivers in a publish-subscribe fashion.
Event-driven microservice The letter E in IDEALS is to remind us to strive for modeling event-driven microservices because they are more likely to meet the scalability and performance requirements of today’s software solutions. This kind of design also promotes loose-coupling since message senders and receivers -- the microservices -- are independent and don’t know about each other. Reliability is also improved because the design can cope with temporary outages of microservices, which later can catch up with processing the messages that got queued up.
But event-driven microservices, also known as reactive microservices, can present challenges. Processing is activated asynchronously and happens in parallel, possibly requiring synchronization points and correlation identifiers. The design needs to account for faults and lost messages -- correction events, and mechanisms for undoing data changes such as the Saga pattern are often necessary. And for user-facing transactions carried over by an event-driven architecture, the user experience should be carefully conceived to keep the end-user informed of progress and mishaps.
Availability over Consistency The CAP theorem essentially gives you two options: availability XOR consistency. We see an enormous effort in the industry to provide mechanisms that enable you to choose availability, ergo embrace eventual consistency. The reason is simple: today’s end users will not put up with a lack of availability. Imagine a web store during Black Friday. If we enforced strong consistency between stock quantity shown when browsing products and the actual stock updated upon purchases, there would be significant overhead for data changes. If any service that updates stock were temporarily unreachable, the catalog could not show stock information and checkout would be out of service! If instead, we choose availability (accepting the risk of occasional inconsistencies), users can make purchases based on stock data that might be slightly out-of-date. One in a few hundred or thousand transactions may end up with an unlucky user later getting an email apologizing for a cancelled purchase due to incorrect stock information at checkout time. Yet, from the user (and the business) perspective, this scenario is better than having the system unavailable or super slow to all users because the system is trying to enforce strong consistency.
Some business operations do require strong consistency. However, as Pat Helland points out, when faced with the question of whether you want it right or you want it right now, humans usually want an answer right now rather than right. Availability with eventual consistency
For microservices, the main strategy that enables the availability choice is data replication. Different design patterns can be employed, sometimes combined:
Service Data Replication pattern: this basic pattern is used when a microservice needs to access data that belongs to other applications (and API calls are not suitable to get the data). We create a replica of that data and make it readily available to the microservice. The solution also requires a data synchronization mechanism (e.g., ETL tool/program, publish-subscribe messaging, materialized views), which will periodically or trigger-based make the replica and master data consistent.
Command Query Responsibility Segregation (CQRS) pattern: here we separate the design and implementation of operations that change data (commands) from the ones that only read data (queries). CQRS typically builds on Service Data Replication for the queries for improved performance and autonomy.
Event Sourcing pattern: instead of storing the current state of an object in the database, we store the sequence of append-only, immutable events that affected that object. Current state is obtained by replaying events, and we do so to provide a "query view" of the data. Thus, Event Sourcing typically builds upon a CQRS design.A CQRS design we often use at my workplace is shown in the figure next. HTTP requests that can change data are processed by a REST service that operates on a centralized Oracle database (this service uses the Database per Microservice pattern nonetheless). The read-only HTTP requests go to a different backend service, which reads the data from an Elasticsearch text-based data store. A Spring Batch Kubernetes cron job is executed periodically to update the Elasticsearch store based on data changes executed on the Oracle DB. This setup uses eventual consistency between the two data stores. The query service is available even if the Oracle DB or the cron job is inoperative.
Event Driven
Resource
Event Driven Architecture
Event Sourcing
Although Event Sourcing has been around for many years, it remains outside the mainstream paradigm of software development. But it is an approach to building software with a long track record of success. With Event Sourcing, we place the highest value on the simple capture of essential business events without attempting to interpret them. By placing business concepts at the heart of our code, we can decouple systems into small services that can be quickly built, changed, and replaced. The resulting systems have single responsibilities and are decoupled from each other, which makes them simple to modify. Event Sourcing can enable us to move faster by supporting rapid experimentation with new perspectives, new user interactions, and new insights into our business.