Closed i-norden closed 3 years ago
KVStore
sIn the state machine, we need to settle on the pattern used to decouple the state commitment and storage concerns. From the ADR-040 proposal, the store buckets consist of:
- SC:
key -> hash(key, value)
: for state commitment we store only the hash(key, value) as the value in the leafs of the state commitment tree.- SS-B1:
key → value
: the principal object storage, used by a state machine, behind the SDKKVStore
interface: provides direct access by key and allows prefix iteration (KV DB backend must support it).- SS-B2:
hash(key, value) → key
: an index needed to extract a value (through: B2 -> B1) having a only a Merkle Path.
A few options for how implement this are outlined below. The best course of action will depend on the order of priorities.
If we want to prioritize SDK user experience, we should wrap this logic in a single KVStore
interface which exposes storage data as KV pairs and only exposes SC data as Merkle root hashes (option #1 below). This only makes sense if users are not expected to need to access or iterate over hashes of individual records or the inverted index mappings. (This is the approach taken in the open SMT store PR.)
If the higher priority is to provide SDK users full visibility and control of all component KV buckets, these should be left as loosely coupled as possible (#2). The SC, SS, and index buckets will only be used in tandem at the multi-store level. The multi-store will then expose these as separate CommitKVStore
s, so that the user is still able to unambiguously access all underlying data, including data and hashed records, and wrap each store in a tracekv.Store
, gaskv.Store
or another layer. However, if the multi-store is planned to be removed in the future, users will then be responsible for managing the new state machine semantics unless a new wrapper type is created.
We suggest a third option if all of the above are considered high priorities: introduce a new interface which expands on KVStore
(#3). Ideally, this will make as much as possible of the underlying store structure available for tooling, while still presenting an SDK interface that is friendly to users.
KVStore
that generates bucket entries and directs writes/reads to both SS and SC.Pros:
CommitKVStore
for each substoreCons:
Set
and Delete
have side effects, rather than simply setting or deleting the provide KV pair they generate two other KV pairs to set/deleteKVStore
s for SC and SSPros:
Set
and Delete
operations remain pureCons:
With option #1 above, the primary issue is not being able to wrap the underlying stores. With #2, the primary issue is that we disrupt the current 1-to-1 mapping of StoreKey
to a KVStore. We can avoid both by defining a new interface and some supporting types:
Pros:
Cons:
StoreKind
enumExample synopsis:
// Wrapper wraps the receiver's underlying KVStores of the specified kind with the provided KVStore
type Wrapper interface {
Wrap(k KVStoreKind, w KVStore)
}
// StateStore is a KVStore that directs reads/write from/to an underlying state commitment, state storage, and state index KVStores
type StateStore interface {
KVStore
Wrapper
}
// Store is an example StateStore, each bucket is a KVStore that can be independently wrapped
type Store struct {
// Direct KV mapping (SS)
ss types.KVStore
// Inverted index of SC values to SS keys
ii types.KVStore
// State commitments layer, LL-SMT KVStore
sc types.CommitKVStore
}
// KVStoreKind enum used to differentiate between the different kinds KVStores underlying a StateStore
// StoreKey remains mapped 1-to-1 with a StateStore, these distinguish between the different substores.
type KVStoreKind int
const (
StateCommitment KVStoreKind = iota
StateStorage
InvertedIndex
)
For all purposes other than wrapping, we can use the StateStore
as any other KVStore
. Methods that need to wrap an underlying store after loading it into the StateStore
will need to use the new methods (e.g. MultiStore.GetKVStore
when tracing or listening is enabled).
Closed in #11
Have this epic encompass the #9892 PR and ics23 proof work
squash https://github.com/vulcanize/cosmos-sdk/issues/8 into this Epic, this will now be Epic 2
We will begin by hooking a new, decoupled, state store (SS) into the SDK. For the time being this will exist alongside the existing IAVL KVStore. When we commit, we save everything to the new SS in addition to the the existing IAVL.
For the SS we add two additional KV buckets:
key → value
: the principal object storage, used by a state machine, behind the SDKKVStore
interface: provides direct access by key and allows prefix iteration (KV DB backend must support it).hash(key, value) → key
: an index needed to extract a value (through: B2 -> B1) having a only a Merkle Path.At this intermediate stage we are not changing what is stored in the state commitment (SC)- the IAVL- we are simply storing two copies: one in IAVL one in the new state storage buckets.