Open ckovorodkin opened 6 years ago
This is a good observation. However, tracking individual block requests and responses would be an overkill. How about increasing default timeout to 2 minutes?
However, tracking individual block requests and responses would be an overkill.
But actually it did, but not use.
Just replace
https://github.com/atomashpolskiy/bt/blob/77315dffded18989c82bb6c8756cb1f4ccf447a5/bt-core/src/main/java/bt/torrent/messaging/Assignment.java#L57
with
long duration = System.currentTimeMillis() - max(started, checked);
and
https://github.com/atomashpolskiy/bt/blob/77315dffded18989c82bb6c8756cb1f4ccf447a5/bt-core/src/main/java/bt/torrent/messaging/TorrentWorker.java#L215
with
shouldAssign = !timeoutedPeers.containsKey(peer);
and for more accurate timeout tracking
to line 64:
and finally
replace
https://github.com/atomashpolskiy/bt/blob/77315dffded18989c82bb6c8756cb1f4ccf447a5/bt-core/src/main/java/bt/runtime/Config.java#L83
with
this.maxPieceReceivingTime = Duration.ofSeconds(10);
(and rename maxPieceReceivingTime
to more actual name)
PS
all my changes that is related to this issue can be seen in my fork: https://github.com/ckovorodkin/bt/commit/6252697470160c5fc52fd6bfdd6f66258a305dbd
BTW, is it ok, that during timeoutedAssignmentPeerBanDuration
Have
messages is still sending to peer (also PEX exchange occurs)?
On the surface, this looks good. But there might be a problem with time-tracking individual blocks in that the sender will not always send the requested blocks with equal intervals. Like we request several blocks at a time (by sending multiple requests from the request queue), the sender may optimize as well and send all requested blocks at once, and this may happen well after the individual block receival timeout has elapsed. I.e. compare the following timelines of receiving blocks from two peers:
Average download rate is the same for both peers (peer Y may even be slightly faster than peer X by sending the blocks after 11 seconds), but with your changes the second peer (Y) will be considered to timeout.
BTW, is it ok, that during timeoutedAssignmentPeerBanDuration Have messages is still sending to peer (also PEX exchange occurs)?
Yeah, I think it's OK. BEP-3 requires to send Have messages to all connected peers, because the most important thing is to keep the view of the state of the swarm up-to-date for everybody.
I think we can just replace the existing option with a new relative metric: max amount of time to receive one unit of data (e.g. 1 MiB) Then calculate session-specific timeout for receiving a piece based on the size of a "standard" piece in the torrent
Metric : minimum download speed
. for example 1.0 KiB/sec
And use it to calculate inactivity timeout
by calculate equation pending block count * block size / minimum dowdload speed
Metric : minimum download speed. for example 1.0 KiB/sec
This is basically the same thing as "max amount of time to receive one unit of data", but differently worded and, when expressed in code directly (for instance, as some Rate
object), harder to serialize/deserialize than a simple number.
And use it to calculate inactivity timeout by calculate equation pending block count * block size / minimum dowdload speed
This is just an approximation, so I don't think it's incredibly useful to constantly perform this recalculation... piece_size * max_data_receive_time
will do just fine (both values should be normalized to the same units, most likely KiBs)
It is more optimal to track smaller amount of data, because it makes possible to quickly detect inactivity. Usually piece size is about 2 MiB, but block size is 8 KiB. Scale is 256:1. When block timeout is 10 sec, than piece timeout is 2560 sec = 42 min. This is too long.
It is more optimal to track smaller amount of data, because it makes possible to quickly detect inactivity.
Optimal from what perspective? Is quickly detecting inactivity of one of the many active peers really that important? I'd rather say that the opposite is true: quickly punishing for eventual pauses is detrimental in the longer run
Instead of short-term micromanagement we eventually would like to maintain long-term statistics (might even persist those between different sessions) for all encountered peers and prefer to assign pieces to peers that proved to be more responsive and timely (while still having some reasonable but not too punishing cap on the receiving time)
Optimal from what perspective? Is quickly detecting inactivity of one of the many active peers really that important?
For real-time streaming applications it is very important. One inactive peer can stop streaming.
On the surface, this looks good. But there might be a problem with time-tracking individual blocks in that the sender will not always send the requested blocks with equal intervals. Like we request several blocks at a time (by sending multiple requests from the request queue), the sender may optimize as well and send all requested blocks at once, and this may happen well after the individual block receival timeout has elapsed.
Just notice: If whole channel bandwidth is utilized by all downloads, than it make sense to download blocks from peer one-by-one. Also, it is preferred behaviour for real-time streaming (while peer get task in block scope, not in piece scope).
Anyway, I plan to implement the simultaneous loading of a piece by several peers in the normal mode (like in endgame, but without random() and duplicates).
Timeout used for whole piece, while it is more appropriate to use it for a small block.
https://github.com/atomashpolskiy/bt/blob/a37ca29f3585d0b34a2fbeb532a92368432dae9c/bt-core/src/main/java/bt/runtime/Config.java#L83
Default timeout is
30 sec
. Let piece size is4 MiB
. In that case peer, that has download speed less than4 MiB / 30 sec = 136 KiB/sec
will be disconnected by timeout.