sofastack / sofa-jraft

A production-grade java implementation of RAFT consensus algorithm.
https://www.sofastack.tech/projects/sofa-jraft/
Apache License 2.0
3.57k stars 1.14k forks source link

proposal: Auto commit mode for applying log iterator #957

Closed killme2008 closed 1 year ago

killme2008 commented 1 year ago

The current log Iterator is used to apply a batch of raft logs to state machine. After the batch is applied to state machine and returns from StateMachine#onApply method, these logs are considered committed.

And while iterating the logs in iterator:

  1. We can call Iteartor#setErrorAndRollback to rollback applied logs and halt the state machine.
  2. Or call Iteartor#commit to commit the logs already applied.

If we want to commit every log while applying, we should call commit method everytime:

while(it.hasNext()) {
    ByteBuffer data = it.getData();
    // apply data to state machine
   ......
   it.commit();
   it.next();
}

I think we can add an auto-commit mode to simplify the code:

// Enable auto-commit mode
it.setAutoCommit(true);

while(it.hasNext()) {
    ByteBuffer data = it.getData();
    // apply data to state machine
   ......
   it.next();
}

After enabling auto-commit mode by it.setAutoCommit(true), the iterator will call commit automatically when calling hasNext() that returns true.

1294566108 commented 1 year ago

Hello, I am interested in this issue. My specific implementation process is as follows:

(1)we can declare a setAutoCommit (boolean status) method in the Iterator interface. image

(2)In the IteratorImpl class, we need to declare two variables of Boolean type. The first is autoCommit, which is used to declare whether to enable the autocommit function, and the second is lastCommit, which indicates whether the previous iterator has been committed. image

(3)So the code for the modified hasNext method is shown in the following figure: Each time the hasNext method is called in the loop, the method will perform additional checks: <3.1> Whether automatic submission is enabled; <3.2> Whether the last loop has been submitted (the lastCommit field defaults to true in the first loop); <3.3> Whether there were errors in the last loop <3.4>Then obtain whether the iterator has any elements that have not been traversed <3.5>Finally, set lastCommit to false, indicating that this loop has not been submitted yet image

(4)We also need to modify the commit method as follows: This means that after we submit, the previous submission status will become submitted image

killme2008 commented 1 year ago

@1294566108 Cool, please create a PR, thanks.

1294566108 commented 1 year ago

Okay, I have submitted the PR, could you please help review it