search

libp2p / notes

libp2p Collaborative Notebook for Research
MIT License
37 stars 4 forks source link

Peers should cache provider information to spread it around the network. #14

Open Stebalien opened 5 years ago

Stebalien commented 5 years ago

After finding new providers for a piece of content, peers should store those providers locally. This:

  1. Speeds up future (initial) find provider queries for the same thing.
  2. Distributes frequently requested provider records around the network.

However, I'm a bit concerned about provider record expiration given a FindProviders call won't actually return when the provider record was initially submitted. This means that popular provider records would live forever.

One solution is to do this (record that some peer provides some content/service) inside the end service itself. For example, when we get a block from a peer, we could locally record a provider record for the peer from which we got the block. This would, incidentally, help us resume downloading a file if we restart.

aarshkshah1992 commented 5 years ago

@Stebalien Why do we want to do this just for the provider information ? Why not for the values passed around as a part of PutValue as well ?

yiannisbot commented 5 years ago

This is a very good idea, which will improve performance and resolution time and which we have discussed briefly with @raulk. In order for this to be efficient, there should be a relation between the time when a block was initially requested and cached and the "expiration time" from the cache. In other words, there should be a link between the position of the provider record in the peer cache (let's call it "provider record cache") and the actual block/data in the peer cache where the provider record is pointing to.

This can only be done probabilistically and should probably also take into account topological issues (e.g., how many nodes can point to an end-peer), but we have shown in [1] that it can be very efficient. (In [1], replace router with peer cache and "EFIB" (ephemeral FIB) with "provider record cache".)

Could the gateway help keep those provider record caches?

[1] A Native Content Discovery Mechanism for Information-Centric Networks, ACM ICN 2017