get_peers is performed but no announce

[Fluturenet]

if I add a log.Printf to annouce.go/beginQuery I get: 20-03-21 21:50:33 announce.go:290: doing dht announce get_peers to [address]

and not a single doing "dht announce announce_peer"

[Fluturenet]

The problem, I think is here:

Multiple transactions can be composed using Select. If the first transaction calls Retry, the next transaction will be run, and so on. If all of the transactions call Retry, the call will block and the entire selection will be retried. For example, this code implements the "decrement-if-nonzero" transaction above, but for two values. It will first try to decrement x, then y, and block if both values are zero.

func (a Announce) nodeContactor() { for { txRes := stm.Atomically(func(tx stm.Tx) interface{} { if tx.Get(a.doneVar).(bool) { return txResT{done: true} } return txResT{run: stm.Select( a.beginGetPeers, a.beginAnnouncePeer, )(tx).(func())} }).(txResT) if txRes.done { break } go txRes.run() } }

[Fluturenet]

Solved:

Invert a.beginGetPeers with a.beginAnnouncePeer

[anacrolix]

The implementation will favour getting peers over announcing to them. If you run out of pending get_peers, it should switch to announce_peer. That may take a while.

[Fluturenet]

but what happens if you are the first one announcing a torrent? this way you will have to wait several minutes before getting online.

[anacrolix]

It does look like it's not very common, here's some values from a long-running instance I have:

"outbound announce_peer queries": 2305,
"outbound find_node queries": 8483,
"outbound get_peers queries": 2519260,
"outbound ping queries": 322392,
"packets read": 5467912,

[Fluturenet]

look the proportions: "outbound get_peers queries": 2519260 "outbound announce_peer queries": 2305

it's 0,0915%

[anacrolix]

but what happens if you are the first one announcing a torrent? this way you will have to wait several minutes before getting online.

You mean that you want to register your interest in the torrent ASAP by sending announce_peer? I believe that at some point you want to terminate your DHT get_peers traversal, and do an announce_peer to a select group of nodes that are closest to your target. Neither logic is in the current implementation: get_peers will starve out announce_peer so long as there are more nodes to contact, and when it does finally stop, it will announce_peer to every single node that returned an announce token.

[anacrolix]

It looks like this is a bug, since it deviates from the BEP, and that a temporary work around would be to favour announce_peer.

[Fluturenet]

i downloaded the dht core of transmission from https://github.com/transmission/dht

if you run the example: ./dht-example -4 56789 router.bittorrent.com 6881 at line 419 change the line to: dht_search(hash, 56789, AF_INET, callback, NULL);

the result: get_peers 75 announce_peers 10 it's 13%

I thing that this library is sending announce_peer too late and this can be a problem for some applications.

[Fluturenet]

This is my log

dht-log.txt

[anacrolix]

Thanks for the informative reporting. I'll get to this when I have time. In the meanwhile, do you think favouring announce_peer would aid in this, or a more complete solution should be the next step?

[Fluturenet]

I've tried to understand what Transmission does: it sets a time window (3 seconds I think) in which makes a batch call of 8 get_peers, and has a way to limit the number of get_peers that are effectuated. This library is too efficient in nailing down every single peer suitable for a get_perr action. As I'm in quarantine .. :-( today I'll try to implement the time-frame method

[Fluturenet]

Look at this !!! Bit torrent techtalks_dht at slide 48 Video on Vimeo fast-forward at 29:40 Perform get_peers until you reach 8 answers, send the announce_peer and wait a certain amount of time.

[anacrolix]

This library is too efficient in nailing down every single peer suitable for a get_perr action.

Do you mean anacrolix/dht is too exhaustive?

[Fluturenet]

32 created a pull request!

[Fluturenet]

This library is too efficient in nailing down every single peer suitable for a get_perr action.

Do you mean anacrolix/dht is too exhaustive?

if you have time look at the video I linked, it's self explanatory the correct behaviour should be to perform a regulated burst of get_peers than announce_peer to the found nodes than sleep for 15 minutes

dht-announce -debug -quiet E2467CBF021192C241367B892230DC1E05C0580E

anacrolix/dht bursts not moderated get_peers and loops forever, it works if you are searching a well note info_hash but doesn't if there are a few peers or you are announcing a new one

[anacrolix]

Thanks, the linked to documents and videos have been very helpful. I've commented on your PR, there are some things in there that are guiding, but not quite idiomatic. I'll run my proposed changes past you.

[anacrolix]

@Fluturenet take a look at https://github.com/anacrolix/dht/compare/issue31.

Old code: *dht.Announce 0xc000138500 of 12bc64fc7688661ac370ea644daf35285c728030 on dht server on [::]:51626 contacted 1424 nodes 12bc64fc7688661ac370ea644daf35285c728030: 280 addrs 167 distinct ips

New code: *dht.Announce 0xc0001ae500 of 12bc64fc7688661ac370ea644daf35285c728030 on dht server on [::]:62263 contacted 32 nodes 12bc64fc7688661ac370ea644daf35285c728030: 197 addrs 161 distinct ips