apache / incubator-pegasus

Apache Pegasus - A horizontally scalable, strongly consistent and high-performance key-value store
https://pegasus.apache.org/
Apache License 2.0
1.96k stars 310 forks source link

fix(duplication): reduce the delay before last mutation is duplicated to the remote cluster #2048

Closed empiredan closed 4 days ago

empiredan commented 1 week ago

https://github.com/apache/incubator-pegasus/issues/2050

As is described by the issue, the problem is that we have to waits 2 ~ 3 minutes (until some empty write gets in) before the last mutation is duplicated to the remote cluster.

The reason is that the last committed decree of the last mutation (i.e. mutation.data.header.last_committed_decree), rather than the decree of the last mutation (i.e. mutation.data.header.decree), is chosen as the max decree that is duplicated to the remote cluster. Instead, the max committed decree should be chosen as the max decree that is duplicated to the remote cluster.

After the optimization, the delay has been reduced from 2 ~ 3 minutes to about 0.1 seconds.