AOF Replay using a Background Task at the Replica - Githubissues

microsoft / garnet

Garnet is a remote cache-store from Microsoft Research that offers strong performance (throughput and latency), scalability, storage, recovery, cluster sharding, key migration, and replication features. Garnet can work with existing Redis clients.

https://microsoft.github.io/garnet/

MIT License

10.37k stars 525 forks source link

AOF Replay using a Background Task at the Replica #783

Closed vazois closed 1 day ago

vazois commented 2 weeks ago

This PR separates AOF enqueue operation from replay. It contains the following

[x] Implementation of an IBulkLogConsumer that performs the AOF replay at the replica side.
[x] Adds throttling of enqueue operations when AOF replay lags beyond a specific amount (i.e. throttle if TailAddress - ReplicationOffset >ReplicationMaxLag)
[x] Ensure re-initialization due to address skipping happens safely while replay consumer is reading from the log.

PS: Fixed a bug related to LRANGE which was incorrectly calling RMWObjectStoreOperationWithOutput instead of ReadObjectStoreOperationWithOutput