cockroachdb / cockroach

CockroachDB — the cloud native, distributed SQL database designed for high availability, effortless scale, and control over data placement.
https://www.cockroachlabs.com
Other
30.13k stars 3.81k forks source link

kv: append log entries outside of raft state machine loop #94853

Open nvanbenschoten opened 1 year ago

nvanbenschoten commented 1 year ago

Extracted from #17500.

After #94165, raft log entry disk writes are asynchronous with respect to the raft state machine loop. However, the (non-durable) engine access for raft log appends is still performed inline. The async storage writes interface permits us to extract all of this work onto a separate goroutine.

This would provide two benefits:

  1. faster state machine loop iteration => less interference between entries => lower latency
  2. larger append batches => more efficient log appends => higher throughput

Jira issue: CRDB-23188

Epic CRDB-39898

blathers-crl[bot] commented 1 year ago

cc @cockroachdb/replication