search

theotherp / nzbhydra2

Usenet meta search
Other
1.23k stars 75 forks source link

Doesn't refresh API limits, or account for them internally #528

Open brianjmurrell opened 4 years ago

brianjmurrell commented 4 years ago

If I hit my download limit on an indexer, nzbhydra2 successfully understands this and reflects it in it's count for the indexer. But it doesn't seem to internally, by say using it's database of downloads, understand when the count of downloads in the last 24h is again below the limit.

I'm assuming it only ever updates the count based on the response it gets from the indexer.

But how does it resolve the situation where it's been told by the indexer that the limit has been hit and therefore never does another search to it. And it must resolve this, somehow otherwise it would just never work for anyone once they hit the limit and clearly it does work for everyone.

So it must just be lazy/lagging in how it resolves it because I can see right now that nzbhydra2 is showing an indexer at the download limit but when I check on the indexer itself, I am only at 70%. I think eventually nzbhydra2 must have to do a search/probe of the indexer and get the new count, but it just seems to be laggy about how it does it.

As I hinted about earlier, nzbyhdra2 could use the downloads database to know when the count is below the limit. Looking at my last $num_limit downloads from an indexer that hydranzb2 thinks is at it's limit, the oldest of those $num_limit downloads is over 24h ago by up to 3 hours. And from the download history, nzbhydra2 is only showing 30% of my limit within the last 24h, so if it were using that history, it would easily see that I am not at my limit.

Just an idea.

theotherp commented 4 years ago

Some indexers return the current values for API and download limits. If they do Hydra uses them. In all other cases Hydra its database to determine when the next API hit should be possible.

It's not a precise science and if the numbers from your indexer and from Hydra differ there may be a couple of reasons, either that Hydra counts some kinds of API hits which the indexer doesn't or because they reset your limits in a matter different from how you configured Hydra.

So, in theory it should already work you describe but there are numerous reasons why it may not...

brianjmurrell commented 4 years ago

So how does Hydra resolve the situation where the indexer is telling it how many hits are left (so Hydra uses that method rather than it's own accounting) but at some point the indexer reports that the limit has already been reached and so Hydra records that and then never does another search because it knows the limit has been hit. If it never does another search (because the limit has been hit) it will never find out from the API that there is new "room" available due to time passing.

theotherp commented 4 years ago

Usually the indexers report when the next API hit is available. See https://github.com/Sonarr/Sonarr/issues/3132#issuecomment-552093398

brianjmurrell commented 4 years ago

Hrm. I see.

Unfortunately the indexer I am using doesn't report *oldesttime. :frowning:

Seems I didn't choose a very good indexer.

theotherp commented 4 years ago

I'm honestly not completely sure how it behaves if that's not set. Which indexer?

brianjmurrell commented 4 years ago

Drunken Slug.

Seems like that might need a hybrid approach, where once at the limit, if a date of next api hit is not given by the indexer then use the search history.

But honestly, I don't plan to be at the limit often if at all, so this is probably a corner case.