async

[btc]

interesting to read in the section "when not to use tokio"

"when reading a lot of files"

https://tokio.rs/tokio/tutorial

[ysono]

This is an interesting point. I didn't know that about lack of async file APIs. I don't intend to work at an abstraction below files. (Eg sqlite seems to manage one big file and prob does some magic to optimize page caching: link. Also there is epoll but I'm not sure how to use it.)

The file IO activities are:

each read request:

• read multiple sstable files

each write request:

• write to commit log (exclusively)

the flushing job:

• read the swapped-out temp commit log file (exclusively)
• write one sstable file (exclusively)

the compaction job:

• read multiple sstable files
• write one sstable file (exclusively)

So it seems we want compaction to be synchronous, blocking any read request. But I believe other operations can be concurrent.

[ysono]

Right now, we're cloneing a lot. This will likely be naturally addressed as part of introducing concurrency.