Is it possible to use this library for DHT router ?

artemnikitin commented 9 years ago

It's not clear about possibility to build a DHT router on this library. It seems that this is possible, because in dht_test.go there is a test TestDHTLocal that seems testing exact same case. But it also looks like test itself is incorrect. If add little bit more logging to it, then output will be

n1 started at port 52361
n2 started at port 52604
n3 started at port 57136
Node n3 found a peer 0: 127.0.0.1:57136
Node n2 found a peer 0: 127.0.0.1:52604

It looks like node found itself as a peer instead of finding at least one more node. Do I understand meaning of the test correctly? Is it possible after all to use this library for building DHT router?

soul9 commented 9 years ago

it's probably not a good idea to connect to the mainline DHT network each time one runs tests. this is why the test boots up a local dht network, on the machine running the test.

artemnikitin commented 9 years ago

Yeah, there is other test called TestDHTLarge that connect to main DHT. But question is not about that ) Question is about possibility of using this library as DHT router like router.bittorrent.com:6881. Test that I mention seems to do exactly that. n1 node created without specification of DHT routers and other nodes using n1 as a router. But at the end they only able to find itself as a peer. Is it correct behaviour or I just don't understand something ?

soul9 commented 9 years ago

oh sorry. yep, looks like they only find themselves. in the DHT itself there is no distinction between routers and other nodes. the "router" is just a node that sits at a pre-defined address and can bootstrap other nodes.

artemnikitin commented 9 years ago

But then it seems that it's not working. Why I haven't been able to find other peers ?

nictuku commented 9 years ago

Sure it is possible to run this as a router. That's what's behind router.magnets.im:6881 which is one of the default routers. It's important to get the terminology right: "nodes" are part of the DHT network, but "peers" are the bittorrent clients.

Other than that I'm not sure what is your question. TestDHTLocal will intentionally only look for themselves and not talk to external nodes. Running a router is easy.

All you have to do is to change the configs a bit, to store more nodes and help the network as much as you can. (peers are not important in this case). Then you have to get your node known in the network. There are a few ways to do that. The most passive one is to just bring up your client and do a few infohash searches. Other nodes will learn about you and some of them (the minority) will add you to their routing table. As time goes by, your node will be added to more and more routing tables and be used more often.

The more aggressive way to become a more popular DHT node (or router, as you say) is to spam the network with endless queries for node infohashes. That's probably wasteful and will probably get you blocked.

The other option, which is what I do, is to hard-code your router's address in a DHT library :-).

If you'd like to add your router address to this library, I would be happy to accept a patch. You'd have to have a stable address (not only a DNS name but also a stable and externally reachable host:port) and it should be running all the time. As long as you give your assures that you'll use it for the greater good, that seems fine by me.

Does this answer your question?

artemnikitin commented 9 years ago

Thanks for an answer )

Maybe it will be better to explain my case. Let's assume there is small DHT network without external access. It's has a router and couple of nodes running locally. Router I create specify config like

config := dht.NewConfig()
config.Port = 3788
config.DHTRouters = ""

And nodes has config

config := dht.NewConfig()
config.DHTRouters = "localhost:3788"

All nodes and router announce some hash. Then I have a some sort of test based on main.go in find_infohash_and_wait folder. I trying to get peers for this hash with method PeersRequest and I can't find one. Do I do something wrong here ? If in this network someone will start downloading this file peers will be find ? In general, if I have a torrent client, who looking in existed router for DHT and trying to download this file by magnet link, will it be able to do so ?

spikebike commented 9 years ago

Any node with:

config := dht.NewConfig()
config.Port = 3788
config.DHTRouters = ""

Will only discover new nodes when they connect to it.

config := dht.NewConfig()
config.DHTRouters = "localhost:3788"

Are you trying to run a small DHT all on the same machine?

I suggest something like running nodeA with config.DHTRouters = "", then nodeB (and any other nodes) with config.DHTRouters = "nodeA:port". That way all nodes will find the nodeA to bootstrap and will then be able to connect to other nodes.

artemnikitin commented 9 years ago

Yeah, I tried to run an experiment on my localhost. Can it be a problem ? I'm doing as you are saying with configuration. Thanks @spikebike for help.

corpix commented 9 years ago

Hi, I have same behaviour as was mentioned.

If there are two or more nodes(introducer.go + node.go + ...) then <-PeersRequestResults on each of this nodes will have a record that points to this node(this node address, port, ...).
If there are two nodes(introducer.go + node.go) then they will never learn each other.
If there are more than two nodes(introducer.go + node.go + node.go) then last two(node.go) will learn about each other.

Is it correct behaviour?

Code I have used: https://gist.github.com/corpix/0e96a477404a903c3ba4

spikebike commented 9 years ago

It's confusion because I don't know which IPs you are using. But say the introducer should intro any peers that connect to it to each other. Try to avoid too many pronouns. In your first case I'm not sure what "this node" refers to.

If you have nodeA = 192.168.1.1, nodeB = 192.168.1.2, and nodeC=192.168.1.3. In introducer on nodeA will discover nodeB when nodeB connects to 192.168.1.1. After which nodeC will learn about nodeB when it connects to 192.168.1.1. After nodeC connects nodeB will also learn of nodeC.

DHTRouter using 127.0.0.1 seems rather weird since you are connecting to yourself and aren't going to learn anything particularly interesting. Or maybe you are trying to run multiple DHT nodes on the same computer for testing?

nictuku commented 8 years ago

I guess we can close this, right?

timesking commented 8 years ago

@artemnikitin did you get local LAN DHT working?

artemnikitin commented 8 years ago

@timesking, unfortunately, no. I switch to other things and stop trying. Sorry for late comment.

nictuku / dht

Is it possible to use this library for DHT router ? #41