microsoft / CCF

Confidential Consortium Framework
https://microsoft.github.io/CCF/
Apache License 2.0
777 stars 211 forks source link

Support K-Value store operations like [.values() - .keys()] #5048

Open Aymalla opened 1 year ago

Aymalla commented 1 year ago

During the development of the data reconciliation application, one of the requirements was to get all the data that was submitted by a member; which is stored on a ccf kv-store; the "kv-store foreach" method was used to implement this requirement.

A better developer experience would be provided if the KV-Store included some features.

Aymalla commented 1 year ago

@achamayou comment

"The issue with pagination right now is that we don't even support ordering, because the CHAMP datastructure we built for the memory constraints of SGX is unordered. We have a ticket and a plan to fix that on our backlog: https://github.com/microsoft/CCF/issues/3088

keys() and values() are possible to add, but quite inefficient if you ever need to access the other half, since you will need to iterate again (storage is row, not column-based). Still seems like a good API ticket to me."

eddyashton commented 1 year ago

This is a great call-out - we have foreach_key and foreach_value in the C++ API, but don't expose them through the JS API. Since the KV type there is based on JS's standard Map type, we could implement the equivalent .keys() and .values() calls.

As mentioned, pagination is harder without ordering, but still something we can consider.