search

tikv / raft-engine

A persistent storage engine for Multi-Raft log
Apache License 2.0
565 stars 88 forks source link

Support Direct I/O #168

Open tabokie opened 2 years ago

tabokie commented 2 years ago

Simple benchmark shows more than 20% throughput improvement from direct I/O.

This task is preferred to be stacked over #165.

sticnarf commented 2 years ago

Besides direct I/O, is there plan to use Linux AIO and provide asynchronous read and write interfaces in the near future?

tabokie commented 2 years ago

No plan yet. But I think it would be trivial to implement once this task (and parallel writing) is completed.

sticnarf commented 2 years ago

For direct I/O, I think one thing we should do before GA is change the file header to occupy the whole 4K block. I think it will be helpful to forward compatibility, so we don't need a format v2 to support direct I/O.

tabokie commented 2 years ago

@sticnarf Sadly, we will default enable it for TiDB Cloud dev tier. Can't change it now.

LykxSassinator commented 8 months ago

Unfortunately, while this enhancement may yield positive results in other scenarios, its impact on TiKV is not as significant.

Based on practical statistics gathered from TiKV across various workloads such as sysbench, tpcc, and ycsb, it has been observed that the typical size during the flushing of raft logs is less than 8kb/s. Consequently, the benefits of introducing DIO are not as impressive as initially expected.

workloads Bytes per second pic
tpcc-1k avg 3KB/s (P99 < 16KB/s) image