ArchiveTeam / seesaw-kit

Making a reusable toolkit for writing seesaw scripts
Other
69 stars 30 forks source link

Change wait time behavior when tracker limit is active #74

Closed chfoo closed 10 years ago

chfoo commented 10 years ago

Change the sleep time behavior between tracker requests when client gets a rate limiting message.

The current sleep time of 30 is causing bad effects on how the tracker is issuing items. Setting a low limit rate in the tracker causes the appears that rate of completion to be slow so tracker admins raise the rate limit. This is wrong because it is simply feeding more and more items into idle threads thus causing more and more requests to the target server. Then once the server overloads, the number of requests coming into the tracker plummets down to 0. The tracker admin then sets the rate limit down to 0. Once the site recovers, a lower limit is used. Repeat ad nauseam.