Goal: After this upgrade, the network will provide distributed and synchronised store services.
Deliverables: (2), (3)
Research tracks: Message Reliability
Estimated date of completion: 2024Q2
An improved version of the Store protocol, marking a crucial increment towards a synchronisation protocol:
introduces the concept of deterministic message hashes to index messages
considers the Store as a key-value store
allows for querying a list of keys (message hashes) from the Store
allows for querying for the full message content (values) of a set of keys from the Store
keeps all previous value-based filtering (e.g. content topic, timestamp) in place
The proposed PR to simplify the Store protocol and use message
hashes as index/cursor, can be used as a starting point.
Store v3 - store synchronisation
Goal: Upgrade the Store service capability in the network from a collection of local, unsynchronised,
semi-centralised (trusted) service nodes to a decentralised service capability in the network with inter-node synchronisation.
Deliverables: (2), (3)
Research tracks: Message Reliability
Estimated date of completion: 2024Q2
Building on Store v3-beta, this version of Store includes basic synchronisation between nodes.
This will probably include:
a protocol/heuristic to resume store services after an offline period
a protocol/heuristic to periodically compare local key-value store with other nodes and find missing keys
a protocol/heuristic to periodically download the messages (values) for missing keys from other store nodes
Note that this can either be
(i) designing a new synchronisation protocol built into the Store protocol
(ii) adapting existing synchronisation building blocks (e.g. a Prolly Tree library) into Store, or
(iii) simply swapping the key-value store itself to a synchronised backend, such as GossipLog
IMO (iii) should be pursued as the preferred option, as far as possible.
SionoiS
commented
3 months ago
Project: https://github.com/orgs/waku-org/projects/16/views/1
Epics
Research Milestone
Summary
An improved version of the Store protocol, marking a crucial increment towards a synchronisation protocol:
The proposed PR to simplify the Store protocol and use message hashes as index/cursor, can be used as a starting point.
Store v3 - store synchronisation
Building on Store v3-beta, this version of Store includes basic synchronisation between nodes. This will probably include:
Note that this can either be
IMO (iii) should be pursued as the preferred option, as far as possible.