search

tikv / raft-engine

A persistent storage engine for Multi-Raft log
Apache License 2.0
564 stars 88 forks source link

Support asynchronously incrementally sync files to disk while they are being written #348

Open overvenus opened 10 months ago

overvenus commented 10 months ago

The idea is borrowed from RocksDB bytes_per_sync. It uses sync_file_range^1 that allows asynchronously incrementally sync files to disk while they are being written. It smoothes out write I/Os over time and reduce fdatasync latency spike as we saw in https://github.com/tikv/raft-engine/pull/346#issue-2059165690 .

Note: The feature proposed here is different from the deprecated bytes_per_sync feature which sync file synchronously.

v0y4g3r commented 10 months ago

A similar issue: https://github.com/GreptimeTeam/greptimedb/issues/3085 Currently we use periodical fsync to alleviate the problem.

Before the periodical fsync the disk ioutil looks like:

image
LykxSassinator commented 8 months ago

THx for your advices. This problems might not be similar to yours. And we've met this problem on Cloud Env when rewriting and fixed it in #347, by introducing bytes_per_sync to limit the maximum flushing bytes. /cc @v0y4g3r

FYI, u can check whether there exists similar syncing problems when rewriting raft logs. And if does, u can check the latest version as your choice.

v0y4g3r commented 7 months ago

THx for your advices. This problems might not be similar to yours. And we've met this problem on Cloud Env when rewriting and fixed it in #347, by introducing bytes_per_sync to limit the maximum flushing bytes. /cc @v0y4g3r

FYI, u can check whether there exists similar syncing problems when rewriting raft logs. And if does, u can check the latest version as your choice.

Do you have any plan to release a new version?

LykxSassinator commented 7 months ago

/cc @overvenus @tabokie PTAL, thx.

We'll do it recently. Before that, u can update your dependency of raft-engine with the latest commit-id.