Waku Research - Post Gen 0 Milestones and Epics

[jm-clius]

After the launch of the public Waku Network (TWN) Gen 0, we want to track our vision for how the network should develop in terms of functional "upgrades". Each upgrade serves as a medium-term milestone consisting of incremental epics. Completed epics should be rolled out to the network as they're completed. This list does not include work in parallel to enable Status Communities to use TWN, which is tracked elsewhere.

Milestone: TWN Store Upgrade (SU)

Goal: After this upgrade, the network will provide distributed and synchronised store services Research tracks: Message Reliability

This track works to upgrade the Store service capability in the network from a collection of local, unsynchronised, semi-centralised (trusted) service nodes to a decentralised service capability in the network with inter-node synchronisation.

Epic SU1: Store v3-beta (message hashes)

An improved version of the Store protocol, marking a crucial increment towards a synchronisation protocol:

• introduces the concept of deterministic message hashes to index messages
• considers the Store as a key-value store
• allows for querying a list of keys (message hashes) from the Store
• allows for querying for the full message content (values) of a set of keys from the Store
• keeps all previous value-based filtering (e.g. content topic, timestamp) in place

The proposed PR to simplify the Store protocol and use message hashes as index/cursor, can be used as a starting point.

Epic SU2: Store v3 (store synchronisation)

Building on Store v3-beta, this version of Store includes basic synchronisation between nodes. This will probably include:

• a protocol/heuristic to resume store services after an offline period
• a protocol/heuristic to periodically compare local key-value store with other nodes and find missing keys
• a protocol/heuristic to periodically download the messages (values) for missing keys from other store nodes

Note that this can either be (i) designing a new synchronisation protocol built into the Store protocol (ii) adapting existing synchronisation building blocks (e.g. a Prolly Tree library) into Store, or (iii) simply swapping the key-value store itself to a synchronised backend, such as GossipLog IMO (iii) should be pursued as the preferred option, as far as possible.

Milestone: TWN Incentivisation Upgrade (IU)

Goal: After this upgrade, the network will provide proof of concept for incentivised service protocols, with a published breakdown clarifying the roadmap to productionisation Research tracks: Secure Scaling, Restricted Run, Protocol Incentivization

Note: This milestone may be significantly disrupted as the overall direction here is still being clarified re topics such as introduction of a Waku Token, decisions about monetary vs reputational incentivisation, etc. Similarly, distributed reputation may soon form a very large part of this effort, but is not listed here as it's unclear where/if it fits the roadmap.

Epic IU1: Store Incentivisation POC

A POC incentivisation mechanism that incorporates POC versions of the three Waku service incentivisation elements:

• a price offer/negotiation mechanism
• a proof of payment system
• a local reputation mechanism to distinguish between "good" and "bad" store nodes

Epic IU2: General Service Protocol Incentivisation POC

This expands POC incentivisation to other protocols:

• consider the most reasonable incentivisation vectors for non-store service protocols, if they need incentivisation at all (filter, lightpush, peer-exchange)
• apply the POC incentivisation elements developed for Store to other service protocols where applicable, delineating alternative strategies where not.

  Epic IU3 - Roadmap towards productionisation

The research and design necessary to come up with a roadmap to productionised incentivisation, including:

1. evaluating the POC incentivisation mechanisms for fairness, security, exploitability and changing tactics where necessary
2. maturing our tokenomics, decisions on a Waku Token, etc.
3. establishing the (possible) role of a global reputation mechanism

Milestone: TWN - Rate-limiting Upgrade (RU)

Goal: After this upgrade, the network will show maturity in how it rate limits, with better dimensioning for bandwidth capping, resource-restricted options and a clear evaluation of what is needed to improve DoS protection. Research tracks: Secure Scaling, Restricted Run, Rate Limiting

Epic RU1: RLNv2

Moving from RLNv1 to RLNv2 to allow better bandwidth dimensioning in the network. This will allow a message allocation per day per registered publisher, providing better statistical guarantees for network bandwidth usage.

See: https://github.com/waku-org/research/issues/22 for more

Epic RU2: RLN in resource-restricted clients

Enabling RLN proof generation and verification within light clients. Different options are set out in https://github.com/waku-org/research/issues/45. Most recently, the promising option of deploying the entire RLN membership tree to a L2 as opposed to a L1 chain, became the key focus of this effort.

Epic RU3: maturing RLN variables/parameters revision (staking, contract/chain, token)

Analyse RLN deployment in TWN Gen 0 and evaluate its DoS protection performance.

• should staking be introduced, especially to improve resilience against adversarial membership registrations?
• should slashing be introduced or does the existing gossipsub scoring method provide enough protection?
• which chain or L2 should we target for memberships?
• if staking is introduced, which token should be used?
• do we need a combination of msg/sec and msg allocation/day rate limiting?

  Further milestones (unscheduled)

The following milestones may be scheduled within the medium term if determined important in the course of other investigations:

• publish integrated peer and connection management strategy for all Waku protocols
• implement distributed/global reputation mechanism
• Waku protocol security review/audit
• Explorations: research into the future of p2p messaging-related topics, such as more advanced routing protocols, episub, peer management strategies, novel ways to incentivise/disincentivise node behaviours, TorPush, etc.

[SionoiS]

My thoughts on SU2, I see 4 separate concept here; query, cache, sync and archive.

• Query: how to ask for the info needed, a solved problem IMO (use SQL)
• Cache: in memory, small and fast messages storage
  • GossipSub already needs a cache of message for deduplication and scoring
  • Could be used as the write head in a Datomic-like DB
  • Sync: How to keep Store node synced without ~consensus~ logical centralization.
  • GossipLog as is.
  • IPLD & Prolly trees with Relay to live sync indexes and ad-hoc connections for longer sync.
  • Archive: system should probably be agnostic. Any DB should do; Codex, Filecoin, Postgres, Sqlite, etc...
  • Most messages should be ephemeral anyway...

[SionoiS]

Thoughs on IU3:

Harberger tax on RLN membership could be the path to a $$$ self-sufficient protocol. Incentives would be aligned, Waku team would strive to make the protocol as good as possible to attract members and generate higher tax revenue.

It removes the one price fits all for memberships while keeping some decentralization and anonymity. The seriousness of the $ investment in TWN would reflects it's usefulness and the ranking of membership investment could act as a reputation system.

The downside would be gas and more smart contract would need to be created. Auctions for memberships, recurring tax, DAO management, etc...

Waku would become serious business :D

[s-tikhomirov]

Harberger tax on RLN membership

Cool idea! Is this somewhat similar to DAI vaults? A contract keeps track of some invariant (in our case - whether I paid the tax), and if it is violated, the asset (RLN membership) goes on sale automatically, and liquidators buy it up.

[s-tikhomirov]

allows for querying a list of keys (message hashes) from the Store

Does "querying a list of keys" mean "give me all keys for the given filters" (time frame, topic, etc), such that I can then selectively query the contents of messages by hash?

It looks to me that designing for SU1 and IU1 should be synchronized: we should think about what the synchronization methods mean in the incentivization context. It is now a bit unclear to me whether we draw a line between a light node querying messages from a full node, and full nodes helping each other sync. In particular, should both these processes be eventually paid for.

On reputation: the line between local and global reputation may be blurry, e.g., the Eigentrust protocol we've been discussing. AFAIU, it is a semi-global protocol: scores are kinda global, but only based on local-ish data. Would it be worth considering such option earlier than in a yet unscheduled milestone?

(Also, fixed a minor typo under SU: "this tracks work" -> "this track works").

Great roadmap overall.

[SionoiS]

Cool idea! Is this somewhat similar to DAI vaults? A contract keeps track of some invariant (in our case - whether I paid the tax), and if it is violated, the asset (RLN membership) goes on sale automatically, and liquidators buy it up.

I guess the mechanism is similar. Not paying tax would result in automatic auctioning of the RLN membership.

[jm-clius]

Weekly Update

• achieved:
  • Significantly refined milestones and epics and started getting feedback from stakeholders
• next:
  • Further stakeholder engagement. Define work needed to improve RFC process.