Multi-stage re-fetch attempts

[zatricky]

When adding multiple nzbs from the same nzb source service, sabnzbd may end up with many items listed as: WAIT 3581 sec / Trying to fetch NZB from https://nzbprovider/path/etc/

I believe this is due to failed fetch attempts or ratelimiting. It appears to wait exactly 1 hour (3600 seconds) before the next attempt. A staged approach for each re-attempt would incur less of a penalty for rate-limited services. For example: 3600 (1 hour) 5400 (1.5 hours) 8100 (2.25 hours) 12150 (3.375 hours)

Separately but perhaps in conjunction: A recognition that the nzbs are all from the same service would also aid in preventing unnecessary fetches. For example, given 100 nzbs but with a ratelimit of 8 per hour, sabnzbd would receive an error on the 9th item. The subsequent 91 items should automatically be delayed for that same interim. In the subsequent fetch attempts, items 9 through 16 may be successful but item 17 may receive an error. The subsequent 83 should then also be delayed.

[shypike]

SABnzbd already tries to play nice with sites it knows. Most sites send no decent feedback or have their own idiosyncratic ways. So most of the time SABnzbd doesn't know the difference between rate-limiting and other problems. I also have few accounts to indexers, so mostly I cannot test.

[zatricky]

Hi, Shypike

Has this been mitigated through some other means? If not, would it be possible to have behaviour as described as the "last resort" behaviour?

Of course, if you simply don't think it is a good idea, that's okay too.

[shypike]

Once in a while, when we get usable feedback from users about certain indexers, we make an adjustment. See history of urlgrabber.py: https://github.com/sabnzbd/sabnzbd/commits/develop/sabnzbd/urlgrabber.py There's no single (or even multiple) fix that solves the whole problem,