Using RwLock.

[ysono]

This is a draft PR that shows how I think mutual exclusion of a background job might look like.

I'm still not sure how to actually make the task run concurrently. (In c++ I'm familiar with the pattern of a class owning a thread and its destructor joining or detaching it, but in rust we need to convince the compiler that self's lifetime exceeds task's lifetime.)

There are two locks within the LSM struct. There should be no deadlock b/c the "flow" of data is unidirectional: head to sstables. Tasks can always 1) update the "downstream" sstables, then 2) update the "upstream" head. We could also ensure all takss lock 1) head 2) sstables in that order. Or we could wrap the whole LSM struct in a RwLock; intuitively I believe this last option is less performant, but who knows until we test it out...

[btc]

We could also ensure all takss lock 1) head 2) sstables in that order.

this is a reliable/standard approach and i support it!

[btc]

I'm still not sure how to actually make the task run concurrently.

how about using async await with tokio? I have never done it before, but my intuition is telling me that a starting point is potentially to look at some idiomatic examples. i recall this mini-redis example project for tokio: https://github.com/tokio-rs/mini-redis

https://github.com/tokio-rs/mini-redis/blob/ebe4e1f33113d8e43a68199d04943b2581b33604/src/server.rs#L274 above is an example of spawning a task

here is how mini-redis spawns background tasks: https://github.com/tokio-rs/mini-redis/blob/ebe4e1f33113d8e43a68199d04943b2581b33604/src/db.rs#L143

i would need to sit down and really internalize the details, but maybe these are a start?