Waku Store v3 protocol - Githubissues

waku-org / research

Waku Protocol Research

MIT License

3 stars 0 forks source link

Waku Store v3 protocol #81

Open jm-clius opened 5 months ago

jm-clius commented 5 months ago

Waku requires an improved version of the Store protocol, marking a crucial increment towards a Store synchronisation protocol.

Amongst other things, this new protocol must:

introduce the concept of deterministic message hashes to index messages
consider the Store as a key-value store
allow for querying a list of keys (message hashes) from the Store
allow for querying for the full message content (values) of a set of keys from the Store
keep all previous value-based filtering (e.g. content topic, timestamp) in place

The proposed PR to simplify the Store protocol and use message hashes as index/cursor, can be used as a starting point.

chaitanyaprem commented 4 months ago

introduce the concept of deterministic message hashes to index messages

Just to clarify the existing Waku-message hashing algo is already deterministic right, so no change is required or expected to be done wrt it. Only the concept of mapping message hash to message is being newly introduced.

jm-clius commented 4 months ago

the existing Waku-message hashing algo is already deterministic right

Yes, it is. If an application crafts a message with a clashing hash, it MUST use the meta field to disambiguate.

One related note is that we cannot currently reconstruct the hash from the stored fields in the archive, because we don't store meta - in other words, if we believe it necessary to in future verify that the stored entry matches its key/hash, we will have to expand our DB scheme to include meta.

chaitanyaprem commented 4 months ago

One related note is that we cannot currently reconstruct the hash from the stored fields in the archive, because we don't store meta - in other words, if we believe it necessary to in future verify that the stored entry matches its key/hash, we will have to expand our DB scheme to include meta.

Interesting point, maybe we should start storing this. This will also address a point made by @kaichaosun in some discussion about validating the contents of a message returned by StoreNode by the requestor.