Key Value API

[jayeshpk1]

I really like fluvio. Thank you. Is it possible to expose the key value api? I am sure there is one that you use under the cover for de-dupe messages.

[sehz]

Hi @jayeshpk1 , yes, we are thinking of doing that. It will just will take some time to figure out API. Any suggestion?

[jayeshpk1]

I can help if you can point me in the right direction

[jayeshpk1]

I would like to see set, get, delete, replace and some batching capability.

[sehz]

Something like this:

pub trait ByteKv {

      async fn get_keys(&self, query: KeyQueryParam) -> KeyStream;
      async fn get_key(&self, key: &Key) -> Option<KeyValue>;
      async fn get_key_values(&self, KeyValueQueryParam ) ->  KeyValueStream;
      async fn update_value(&self, ,key: &Key, value: Value) -> Option<Value>:

      // key can no longer be allowed to update
      async fn disable_key(&self,key: &Key);

      // no longer can be update nor retrieval,  it would be still in storage
      async fn forget_key(&self, key: &Key);

      // make sure it is flushed to persistent
      async fn flush(&self);

}

Key and Values are arbitrary byte values. If you want higher level with type abstraction then wait for SDF. Anyway, this is first sketch.

[jayeshpk1]

Yes. That will work. Thanks.

[digikata]

You might also look at uses of the fluvio-kv-storage crate. Internally, we use that for things like offset management in fluvio-spu/src/kv/consumer.rs. It's a pretty focused implementation, so you would have to evaluate if it fits your dedupe case.