add an optional durability level to WriteOptions

[steveyen]

moss defaults to asynchronous batch persistence, but users should be able to ask for synchronous batch persistence by just flipping on an option.

[bt]

Wondering if there's any planned updates on this issue?

I've had a look at the code for moss but I think it needs some refactoring before such option can be added.

[hisundar]

Hi @bt , moss has a bunch of EventCallbacks which notify the caller on various events in moss. https://github.com/couchbase/moss/blob/master/api.go#L256 One of these is the EventKindPersisterProgress which notifies the caller once the background persister is done writing out data to disk. The callback can then stall the app to wait on it. Here is one basic example on how it can be used, https://github.com/couchbase/moss/blob/master/store_test.go#L1975 It's not too fancy, but perhaps works for your usecase?

[mschoch]

@hisundar correct me if I'm wrong, but this approach can't be used in a general way when you're executing multiple concurrent batches. The reason is that the dirty bytes may never go to 0, or if it finally does, it's only after all the batches in flight have finished. So, it would be safely durable, but not very usable for an application in practice.

[bt]

Personally I feel that this is more like a hack than it is a solution.

Perhaps we can consider a refactor to make it cleaner?

[hisundar]

You are right @mschoch this approach can't be used to concurrently insert and wait for persistence at the same time, well at least not trivially. But it can be used for a usecase which does a bunch of batched inserts first, then waits for persistence on all of them, before the next set of batched inserts and so on. @bt as the title suggests, I believe the idea is to add a cleaner option of doing this possibly via the WriteOptions. I just wanted to suggest something that works today for a simple use-case.