Upcoming DHT redesign/rewrite

[nazar-pc]

We've started from WebTorrent DHT, which in turn is based on Mainline DHT. This allowed us to prove that such a system can work in browser.

But besides some legacy unnecessary stuff it is also not robust and not resistant to attacks and we need to design an alternative specification and then implementation specifically for Detox project.

Few major issues we need to resolve:

• Resistance to Eclipse attack (protect nodes from being surrounded by adversary-controlled nodes only or mostly, needs proper quantifying)
• Resistance to Sybil attacks (prevent adversary from deploying excessive numbers of nodes to the network)

We already use public keys instead of arbitrarily selectable IDs, which is the first right step. Now we need to borrow from other papers or implementations or invent ourselves some kinds of crypto quizzes that regular nodes will be able to solve, but will be hard enough for an attacker to handle.

Few more specific thoughts:

• DHT keypair's public key hash (Blake2b in out case as we use it in other parts of the system) should start with certain number of zero bits (can be adjusted to the network conditions) like suggested in S/Kademlia paper
• Some kind of ongoing, likely time-related quiz that all active nodes should solve in time, this might prevent an attacker fro maintaining too many nodes in the network at the same time
• Find a clever way to force an attacker to be useful to the rest of the network against their will (meaning that otherwise they will violate the protocol and will be rejected by valid peers)

This could increase barrier for joining the network as well as being an active part of the network.

Also comparing to Mainline DHT we'll need to reduce number of methods and only allow mutable keypair/signature-based keys for data stored in DHT (also hard and/or soft size limit should be reevaluated, Mainline DHT uses 1000 bytes) limit.

Useful links:

• Kademlia - http://www.scs.stanford.edu/%7Edm/home/papers/kpos.pdf
• Mainline DHT: http://bittorrent.org/beps/bep_0005.html
• Storing arbitrary data in Mainline DHT: http://bittorrent.org/beps/bep_0044.html
• S/Kademlia: https://pdfs.semanticscholar.org/3165/2823ca71520038773346b6e5bbfadc5c8419.pdf
• Chord: https://pdos.csail.mit.edu/papers/chord:sigcomm01/chord_sigcomm.pdf
• "Salsa: A Structured Approach to Large-Scale Anonymity": https://www.freehaven.net/anonbib/cache/Salsa.pdf
• "ShadowWalker: Peer-to-peer Anonymous Communication Using Redundant Structured Topologies" : https://www.freehaven.net/anonbib/cache/ccs09-shadowwalker.pdf
• "NISAN: Network Information Service for Anonymization Networks": https://www.freehaven.net/anonbib/cache/ccs09-nisan.pdf
• TON DHT: https://denull.ru/telegram/ton-tech.pdf

[nazar-pc]

Underlying framework for DHT will likely be generic and implemented as standalone project https://github.com/nazar-pc/es-dht, while @detox/dht will use it to implement domain-specific version.

[nazar-pc]

Initial implementation based on ES-DHT released as https://github.com/Detox/dht/releases/tag/0.15.0