cockroachdb / cockroach

CockroachDB — the cloud native, distributed SQL database designed for high availability, effortless scale, and control over data placement.
https://www.cockroachlabs.com
Other
30.19k stars 3.82k forks source link

raft: eliminate log scan on campaigns #131559

Open pav-kv opened 2 months ago

pav-kv commented 2 months ago

The config change implementation in raft package requires a candidate to check whether the raft log has unapplied config changes. This is implemented as a log scan, which potentially reads from log storage.

We would like to eliminate cases of uncontrolled IO from inside raft, which this scan classifies as.

One way to eliminate this scan:

This change probably requires a migration: upon startup, scan all raft logs (alternatively, limit the scan to unapplied entries only), and initialize this variable in storage. This change would be best bundled with any other changes that need a full-scan raft log migration.

Jira issue: CRDB-42602

pav-kv commented 1 week ago

Invariant (implied from the Append invariant): in a log, there is at most one config change at index > commit index.

This is convenient. We can maintain the last <= 2 config change indices (only one if all are committed). The penultimate one is guaranteed to be <= commit index, and the last one may or may not be known to be committed. These indices can be updated incrementally upon each log append.