Tokio file IO has significant overhead, since each syscall like write needs to spawn a task and perform a thread context switch. This was measured to about 8 µs on my MacBook, which quickly adds up with many small writes.
We should consider using tokio-epoll-uring for file IO instead, to avoid this cost.
This may be mitigated by improved batching instead (#9689), but there may be workloads for which it's still beneficial. It might also help with concurrent writes across timelines, since it would likely reduce pressure on the Tokio scheduler.
Tokio file IO has significant overhead, since each syscall like
write
needs to spawn a task and perform a thread context switch. This was measured to about 8 µs on my MacBook, which quickly adds up with many small writes.We should consider using tokio-epoll-uring for file IO instead, to avoid this cost.
This may be mitigated by improved batching instead (#9689), but there may be workloads for which it's still beneficial. It might also help with concurrent writes across timelines, since it would likely reduce pressure on the Tokio scheduler.