libp2p feasibility study for Waku routing

[oskarth]

Waku libp2p routing feasibility study

Background

Currently the the Status app is using Waku (and/or Whisper) in the app. This is running on devp2p with gossip/flooding routing based on Whisper.

Problem

1. Current routing scales poorly between full nodes due to excessive amplification factors (every node receives every message, often more than once)

2. Running on devp2p as opposed to libp2p limits us in several ways:

   • means we can't run in a browser (websockets/webrtc)
   • can't get access to more efficient transports such as QUIC
   • whole community, incl Eth and IPFS, moving to libp2p - all improvements are there
   • censorship-resistance with traffic obfuscation
   • devp2p rlpx encryption broken

3. Lack of incentive to route, leading to unsustainablity (modulo altruism / higher-order incentives) and centralization (Status cluster)

Solution

We want a routing mechanism that is decentralized, incentivized and scales.

The goal of this specific issue is to understand options and limitations for moving Waku to libp2p. We do this by studying the feasibility of two promising candidates:

• GossipSub (unstructured, spec mature)
• Kademlia (structured, well-understood)

Optionally, a pub/sub interface on top of Kademlia to have more harmony with existing app requirements.

We want to understand and isolate properties and constraints such as:

• practical blockers for moving Waku to libp2p
• interface harmony (topic vs only A-B routing for app requirements)
• ability to build pub/sub on top of kad
• scalability and spam resistance
• implications for resource restricted devices and light connection mode
• different modes of kademlia (forward/classical)
• ability to do accounting and incentive settlement
• other considerations (re encryption, etc)
• potential to decouple encryption from routing/messaging layer (secure chat vs whisper double concern)
• bandwidth/privacy (due to gossipsub decoupling)

Acceptance criteria

• Study of above with clarity of above issues
• In cases where there are blockers and/or more clarity is needed, isolate these problem descriptions
• Write up of said feasibility study
• Plan for how to proceed engineering-wise

Notes

Related reading:

• Gossipsub libp2p spec
• Kademlia paper
• Kademlia xlattice spec

[oskarth]

Rough initial draft https://notes.status.im/waku-libp2p#

[oskarth]

Brief update: nim-waku code here with experimental v2 https://github.com/status-im/nim-waku/

Quite a bit to go. Will split up above issues into separate GHIs

[oskarth]

While we could do more work here, I'd consider this done for now https://discuss.status.im/t/waku-version-2-pitch/1776