Open aschmahmann opened 4 years ago
ipfs/go-ipfs#7169 is related.
@makeworld-the-better-one there may be some relationship between these two issues.
Note though that this proposal assumes that one network is a subset of another (e.g. I first check Yggdrasil for data, then the regular network) and the proposal will cause problems if this is not true (e.g. some nodes use Yggdrasil, some use CJDNS, some use both).
Ah okay. Yeah, in the issue I linked it's referring to separate DHTs and networks, not nested ones.
A number of large changes to the DHT (e.g. AutoNAT based automatic switching between client and server modes) have occurred in recent months and there are many more ahead (e.g. #558 #559 #584). We need to figure out how to do protocol upgrades in a smooth way, and ideally one with the following properties
1) Small groups can attempt an upgrade without disconnecting from the larger network (i.e. no large coordination costs to propose a network upgrade). 2) Can use this strategy to establish an open testnet that can be upgraded without waiting on an official release 3) Minimal latency addition to queries 4) Minimal machine/network resource utilization to be part of a testnet
Proposal
Nest DHTs inside of each other. For two DHTs A and B insist that every member of A is a member of B (i.e. A is nested inside of B, such as A =
/testnet/kad/2.0.0
, B =/mainnet/1.0.0
). Then we can go through the following procedure (usingGetClosestPeers
as an example, but applies to the other DHT query types as well)1) Query
/testnet
for the closest nodes. 2) Query using the/mainnet
protocol where the lookup procedure is seeded from the results of the/testnet
query (instead of/in addition to the/mainnet
routing table)/testnet
are only clients of/mainnet
, if so then there needs to be a transfer step where we ask the closest nodes from the/testnet
who the closest nodes they know about in/mainnet
are. If/testnet
servers are also/mainnet
servers then this happens automatically.Puts
Depending on what we want we could do Puts in a couple different ways, such as: 1) Put to both networks (good for search performance in the new network) (my preference) 2) Put to only the main network (assuming that servers in
/testnet
are servers in/mainnet
)/testnet
once it is big enough to automatically get data by virtue of being a server in the main network (good for reducing storage on the network)Bonuses
Having this setup also gives us some freebies like paving the way for allowing for multiple DHTs with different validators so that a network can have a shared base DHT (e.g. for finding other peers) and then application-specific DHTs layered on top (e.g. for finding/storing application specific content).