Open IbrahimMNada opened 2 days ago
This would be a nice feature, but I'd prefer to use something like Redis pub/sub directly assuming that's what we are using for the L2 cache.
But like this it would make every one coupled with Redis,
I mean we could have it as an optional provider , but we need some default with no extra setup that's why I suggested web-hooks
We could have multiple providers like but not limited to 1-redis pub/sub 2-Event buses 3-Event signal R hub
Hello , @mgravell
could you please give us your invaluable input on this ? to be honest I'm eager to help
Background and motivation
In a multi-replica environment utilizing hybrid caching (in-memory and out-of-process), cache desynchronization between nodes can occur because there is no built-in mechanism to synchronize in-memory caches across nodes behind a load balancer. This results in inconsistent cache states, reducing the reliability of the system.
This proposal addresses the problem by introducing an event-driven mechanism to ensure cache synchronization across nodes.
Problem Context
Hybrid caching involves two main components:
When a cache is reset in one node, other nodes do not get notified, leading to cache desynchronization across the system.
Problem Statement
The current hybrid caching model does not offer a built-in mechanism to notify all nodes about an in-memory cache reset, resulting in inconsistent cache states between nodes in a multi-node environment.
API Proposal
Proposed Solution
Overview
Introduce a Publisher-Subscriber model using webhooks, event queues, or other notification mechanisms to propagate cache reset events to all nodes using the hybrid cache. This model will allow one node (the Publisher) to notify other nodes (Subscribers) when a cache reset happens, ensuring synchronization of the in-memory cache across all nodes.
Key Features
Webhook-based/Callback mechanism: Each node registers as a Subscriber to receive notifications when cache resets happen. The node initiating the reset acts as the Publisher.
Retry strategy: In case of a failure in notifying a node, the system retries the notification, ensuring robustness in cache synchronization.
Multi-provider support: While webhooks are the default, the design allows support for other messaging systems like
event queues
,SignalR
, etc.API Changes
Add a
CacheResetNotification
class:API Usage
Alternative Designs
Polling Mechanism: This was dismissed due to inefficiency and increased load on the nodes.
Risks
We need to consider network flooding or over requesting between nodes , so we can define a sync period between nodes/keys count thresh hold or something like this