search

pillone / usntssearch

NZB Metasearch engine
pillone.github.io/usntssearch
263 stars 79 forks source link

Offset parameter is not respected #110

Open markus101 opened 10 years ago

markus101 commented 10 years ago

Hello,

I'm one of the developers on NzbDrone (https://github.com/NzbDrone/NzbDrone) I have had a number of users experience issues with Megasearch due to it not respecting the offset parameter for the newznab API. The offset parameter allows for paged results, so a search via NzbDrone will allow for all releases of that season to be fetched from the indexer in a single search (multiple calls to the API).

The current response NzbDrone gets from Megasearch is the same results over and over, which causes done to get stuck in a loop. some improvement needs to be done on the NzbDrone side, but changes there would only some the looping issue, but paging would still not be possible.

Please let me know if you have any questions or concerns.

pillone commented 10 years ago

Hi there. I always ignored the offset param due to the marginal utility of it. Statistically speaking is it very unlikely that you will find something in the next 200 results.

If it is something absolutely needed I can implement it in the next ver.

Please consider also that megasearch does not accept parallel reqs. It was a choice for not overloading all the configured servers.

markus101 commented 10 years ago

Hey, thanks for getting back to me. This is going to be a bit long winded, but its the best way I think I can get my point across:

The reason we use the offset is this: Perform a full season search, which contains 25 episodes and has been released on Bluray, but not DVD and the user wants DVD or SDTV (408p).

Assuming each quality has all 25 episodes and the following qualities, Bluray 1080p, Bluray 720p, WEBDL 720p, HDTV 720p, SDTV.

The top 100 results would contain all the BR (720p and 1080p) releases (because they came out at the same time), leaving 50 episodes left, which would be split in 3, so approx 16 episodes for WEBDL 720p, HDTV 720p SDTV, leaving the first 9 episodes of that season unavailable in WEBDL 720p, HDTV 720p and SDTV, which means to fill those in the user needs to perform 9 additional searches to get them.

1 additional search would would grabbed the next 100 results from the indexer and resulted in 2 total queries to the indexer for one click instead of 10 queries and 10 clicks. This example is a bit exaggerated, but even with 20 episodes a season with more qualities (1080p releases in WEBDL + HDTV as well as DVD) this is still an issue.

In our testing all the results are retrieved within the first 3-4 queries and it makes the user happy because its all seamless to them.

The perfect solution would be to support this, alternatively it can be worked around by not returning results when the offset is over X and/or limiting the number of calls drone makes before aborting.

Thanks for considering.

Mirabis commented 10 years ago

Any word about this?

TRaSH- commented 10 years ago

i love this program but i also noticed not much activity in here, it's a shame for a great program like this. running in to the same issue