search

ipfs / go-bitswap

The golang implementation of the bitswap protocol
MIT License
216 stars 112 forks source link

Predict peers most likely to have blocks #14

Closed eingenito closed 5 years ago

eingenito commented 6 years ago

Problem Right now we don't make the best use of information we have about our historical bitswap interactions with peers when devising a set of peers to ask for blocks.

PR #8 has some important background information particularly this comment.

Proposal Add behavior to bitswap that tracks received blocks and the peers they came from, which can subsequently be queried for an (ordered?) set of peers deemed the best candidates to ask for a given block.

eingenito commented 6 years ago

A more concrete implementation plan for this feature emerged from conversations at the go-ipfs hack week in London.

There's already some acceptance of this concept as a predictor service, driven by bitswap. Now there's a proposal that switches that relationship and places this logic between the DagService and bitswap (or other protocols, such as graphsync).

The new interface would be a peer of the SessionExchange, but for graphs. Both would roughly expose ask for x or []x where 'x' is currently CID for SessionExchange and would be a simplified set of graphsync expressions in the case of the new DagExchange (or GraphExchange?). They would drive fetchers which currently would just be bitswap and which would conceptually expose ask peer x for y[].

Here is a diagram from our discussion:

img_20181017_1402387

Our initial work would focus on first splitting bitswap so that it could be driven by different 'exchanges', and then implementing the new DAG oriented exchange on top of that lowest layer of bitswap that just fetches data.

hannahhoward commented 5 years ago

@eingenito my feeling is this plan has been superceded by the ongoing work to improve sessions, and the emergence of Graphsync + go-ipld-prime as protocols. IOW, I think rather than try to build dag smarts into bitswap, we should just continue optimizing it as a block exchange protocol, and then utilize graphsync as it continues to evolve as the ideal dag exchange protocol. I am inclined to close this issue unless you think otherwise.

eingenito commented 5 years ago

Agreed.

Stebalien commented 5 years ago

:+1: This may be what you're saying but we could even have graphsync drive bitswap directly in the future (i.e., we ask for some subdag, a peer gives us most of it but tells us it doesn't have some set of nodes).