search

ipfs / kubo

An IPFS implementation in Go
https://docs.ipfs.tech/how-to/command-line-quick-start/
Other
16.03k stars 3k forks source link

Support publishing self-records in LAN DHT even when WAN DHT is available. #7168

Open makew0rld opened 4 years ago

makew0rld commented 4 years ago

As I mentioned in #7109, I am part of @tomeshnet, and we use IPFS on our experimental mesh network nodes. There are some new changes in the upcoming v0.5.0 release that will affect us, that I discussed with @Stebalien and @willscott in that issue, eventually leading me to create this feature request.

The only question I have now is whether there's a way to disable the setting that means nodes "will only publish records (provider and IPNS) to the WAN DHT". In a situation where a node is connected to the Internet and CJDNS, for example, we'd want them to still be publishing records to the LAN (CJDNS). Or more specifically, we'd want them to publish any of their own records to the LAN, but not WAN records that a CJDNS-only node couldn't access. -- me

@Stebalien informed me this is not possible, and so I'm asking if this can happen, through a config option, as well dynamically through an ipfs command.

To be clear, when this option is enabled on a node, then anytime a node is publishing records on the WAN DHT for blocks/IPNS addresses that it has, it should also publish those records on the LAN DHT. It should not publish records that point to WAN nodes.

The endgoal of this is that if you have node A that is connected to the Internet and a private mesh network like CJDNS, and A adds a file to IPFS, then B which is only connected through CJDNS, should be able to retrieve the file, because A published the record for the file on the LAN DHT that B was a part of.

makew0rld commented 4 years ago

@Stebalien @willscott Do you think this could happen?

willscott commented 4 years ago

The main question in my head is if there's a good heuristic (e.g. number of lan peers, diversity of address space they're in, or something else) that should prompt that behavior to happen automatically.

Expect the behavior here to change again fairly soon with regards to https://github.com/libp2p/go-libp2p-kad-dht/issues/616 - There's certainly a few needs to do better than the current implementation with dealing with the reality of multiple active DHTs.

makew0rld commented 4 years ago

Even if there is, I think it still makes sense to have this able as a config and command line option, so the user can decide as well.

darkdrgn2k commented 4 years ago

I will say that to some degree its a moot point too.

One issue with LAN ONLY based IPFS network is you may not be able to reach a BOOT STRAP node to start peering.

Although we have an out of band mechanism @tomeshnet to find neighboring nodes it only works if the out of band script is installed on every IPFS node on the LAN since it must reveal the node's ID to establish a peer.

I'm not sure if mDNS was ever fixed/working but unless it is or another mechanism is put in place a LAN ONLY IPFS network in the would be a non starter.

makew0rld commented 4 years ago

In this case, LAN = not public Internet. It doesn't just mean link-local. For IPFS, all of CJDNS and Yggdrasil are considered "LAN", I've confirmed this with the developers in other threads.

darkdrgn2k commented 4 years ago

I don't know if this is the same in IPFS but with GIA the Yggdrasil's "client" network (200::/8) is not considered LAN. (this causes other problems)

If you cannot reach an IPFS bootstrap node, or don't have the IP/ID of one to peer with or boot strap yourself (ie tomesh stack for CJDNS where we added one) then a LAN node wont peer and therefore not need DHT.

makew0rld commented 4 years ago

Yggdrasil is considered LAN in IPFS, see this issue.