search

probe-lab / go-kademlia

Generic Go Kademlia implementation
Other
18 stars 4 forks source link

Feat: Incorporating Geographical Distance or Network Latency in DHT #75

Open wei-b0 opened 1 year ago

wei-b0 commented 1 year ago

Hi, I've been recently discussing a specific use-case scenario with the libp2p community that revolves around the concept of locality in DHT (Distributed Hash Table) and content routing. In our particular use-case, the geographical distance or network latency between nodes is of utmost importance and it's a key factor that we cannot afford to overlook.

Currently, as far as I'm aware, the libp2p.contentRouting.findProviders(CID) returns the closest nodes based purely on XOR distance between the CID and the peer-id. While this is great for most applications, it lacks the locality-preserving feature that would allow us to account for geographical distance or network latency.

It was suggested to me by yiannis in our conversation that these thoughts could be added as a feature request for the ongoing DHT refactoring work. Hence, I'm opening this issue. Thanks,

wei-b0 commented 1 year ago

@yiannisbot

iand commented 1 year ago

It should be possible to add latency as a signal as part of the routing table refresh algorithm (in addition to reachability). However this can leave the DHT open to eclipse attacks since an attacker can bring up enough new DHT nodes in close proximity to you to fill your routing table.

The existing go-libp2p-kbucket repository implements a peer diversity filter and we will likely incorporate something similar to ensure that buckets have a mix of peers. This is issue https://github.com/plprobelab/go-kademlia/issues/49.

guillaumemichel commented 1 year ago

This repository is meant to provide the basic tools to build a DHT implementation. If you want to add ping distance in the routing algorithm, you can create a custom routing table taking ping distance into account when selecting the best peers to keep track of. You may also want to modify the server behavior when responding to lookup requests. Note that this repository isn't ready for production yet, and interfaces will certainly change.

I would like to add latency in the network simulator (in FakeRouter) so that developers can start experimenting with locality aware DHT implementations.

On the very long term, we would like IPFS to take ping distance into account during the routing process, but this isn't the immediate next step