Sending an operation doesn't give the server enough info to know you've acked up to the server order of that operation, so the next sync can (and does) return that operation to you (maybe just for push/pull mode?)
The response to a replica operation is kind of wasted with a simple ack right now
Idea:
Keep push/pull replicas alive in memory for some period of time. Enqueue op-re messages for those replicas in their message queue. When that replica sends a request to submit an op, respond with the ack and the drained message queue. The replica then sends a final ack for the latest op-re server order. This functions as an on-the-fly sync, allowing push/pull replicas to sync more continuously as long as they are sending ops.
Two concepts:
Idea:
Keep push/pull replicas alive in memory for some period of time. Enqueue op-re messages for those replicas in their message queue. When that replica sends a request to submit an op, respond with the ack and the drained message queue. The replica then sends a final ack for the latest op-re server order. This functions as an on-the-fly sync, allowing push/pull replicas to sync more continuously as long as they are sending ops.