[DHT] Prefer low latency peers to reduce query times

jacobheun commented 4 years ago

Coming Soon

aarshkshah1992 commented 4 years ago

@jacobheun @Stebalien

Do we have a writeup somewhere of what we are trying to accomplish here and when it would deemed to be satisfactorily completed ?

jacobheun commented 4 years ago

I think we should wait on this, and potentially look at for IPFS 0.7 once we have more information.

The general idea is to avoid scenarios where we are dialing distant nodes.

In our recent Testground runs for the DHT, we saw about 1.2 - 1.4 seconds for a single node DHT query at a simulated 100ms latency. The TCP connections currently take ~6 round trips to query, which includes the TCP handshakes, crypto and multistream negotiation.

The quickest way to reduce query times is lower the number of hops we need to perform. If we can finish querying in 3 hops, we're in the range of roughly 3.6 - 4.2 seconds. Reducing hops further with accelerated table lookups is a big win.
Reducing is the number of round trips for the query is another thing to look at, which is where things like QUIC and multistream/multiselect 2 will come in. I reduction in 1 round trip for a 3 hop query at 100ms of latency would save us 600ms (RT 2 latency * hops). 3.0 - 3.6 s query times.

Then we have latency, which is this issue. Roughly speaking, let's assume we are able to isolate a query completely to our local latency region. We might be able to get an average latency of say, 50ms. This would drop our original times from 3.6 - 4.2 s, to 1.8 - 2.1 s, assuming we don't need to query outside of the latency region. On the other hand, if we don't account for latency, we might end up querying peers with a 200ms or more latency to us. If our average query latency is 200ms, we double our original query times, 7.2 - 8.4s.

If we use latency overlays we can query our local ring first, and work our way out if we can't find the content. I think this would be a good topic to start discussing with @petar, @aschmahmann and likely @yiannisbot. There was some discussion of this earlier this year. I think targeting a design and test strategy for this as part of the IPFS 0.6 work and potentially slating it for IPFS 0.7 would be a more reasonable time frame given the complexity.

aschmahmann commented 4 years ago

Thanks for the clarification @jacobheun. If this is about effectively having latency groupings/levels then this is definitely not in the implementation cards for IPFS 0.6, but allocating time for planning/investigation seems reasonable.

I think we should probably tackle https://github.com/libp2p/go-libp2p-kad-dht/issues/546 and https://github.com/libp2p/go-libp2p-kad-dht/issues/566 before going to far into this though since those should help bring us down to 2-3x hop time (we expect ~3 hops, but the first hop should be to peers are connected to, which if we're using TCP + multistream/1 should be 1/6 the time of the subsequent hops).

jacobheun commented 4 years ago

I think we should probably tackle libp2p/go-libp2p-kad-dht#546 and libp2p/go-libp2p-kad-dht#566 before going to far into this though since those should help bring us down to 2-3x hop time (we expect ~3 hops, but the first hop should be to peers are connected to, which if we're using TCP + multistream/1 should be 1/6 the time of the subsequent hops).

Agreed, let's tackle the quicker stuff in 0.6. Both of those are now in the 0.6 milestone, I will move this one to 0.7

libp2p / go-libp2p-kad-dht

[DHT] Prefer low latency peers to reduce query times #807