microsoft / LSKV

A Ledger-backed Secure Key-Value store (LSKV), built on the Confidential Consortium Framework (CCF)
https://microsoft.github.io/CCF
MIT License
33 stars 6 forks source link

Add dynamic public prefixes through constitution #188

Closed jeffa5 closed 1 year ago

jeffa5 commented 1 year ago

Previously the maps for kv would be either all private or all public. This makes it more dynamic, moving the option into a constitution-managed map.

Every key is kept in the private map, keys that match the public prefix will also be kept in a public map which is viewable in the ledger. By default there are no public prefixes and so everything is private.

Fixes #179

Code caveats:

jeffa5 commented 1 year ago

I think it would be much preferable to have this as constitution-first rather than going with this intermediate step but at least this shows a potential way forward.

jumaffre commented 1 year ago

I think we should add a comment somewhere that this is a bit of a hack, and that the nicer solution to avoid duplicate storage would be to add an iterator API to the CCF kv::Map so that we can foreach across two maps (private and public).

github-actions[bot] commented 1 year ago

Plots from benchmarking at 29b2e0d6fedc70053a09e2aa3db49865b607bb35

Available from here

etcd/achieved_throughput_bar-vars-None-nodes.jpg
etcd/ecdf-latency_ms-None-nodes-vars.jpg
etcd/scatter-start_ms-latency_ms-None-nodes-vars.jpg
etcd/target_throughput_latency_line-rate-None-nodes-vars.jpg
etcd/throughput_bar-rate-None-nodes-vars.jpg
k6/achieved_throughput_bar-vars-None-func.jpg
k6/commit_checks_ecdf-committed_count-None-None-vars.jpg
k6/commit_latency_ecdf-commit_latency-None-None-vars.jpg
k6/ecdf-latency_ms-None-func-vars.jpg
k6/scatter-start_ms-latency_ms-None-func-vars.jpg
k6/target_throughput_latency_line-rate-None-func-vars.jpg
k6/throughput_bar-rate-None-func-vars.jpg
perf/achieved_throughput_bar-vars-http_version-nodes.jpg
perf/ecdf-latency_ms-http_version-nodes-vars.jpg
perf/scatter-start_ms-latency_ms-http_version-nodes-vars.jpg
ycsb/achieved_throughput_bar-vars-None-None.jpg
ycsb/ecdf-latency_ms-nodes-operation-vars.jpg
ycsb/scatter-start_ms-latency_ms-nodes-operation-vars.jpg
ycsb/target_throughput_latency_line-rate-None-nodes-vars.jpg
ycsb/throughput_bar-rate-nodes-operation-vars.jpg
jeffa5 commented 1 year ago

Needs to wait for a new CCF release so that we have the cose tooling