search

Metron-Project / Simyan

A Python wrapper for the Comicvine API.
https://pypi.org/project/simyan
GNU General Public License v3.0
11 stars 2 forks source link

Search Endpoint Usage #114

Closed themadman0980 closed 2 years ago

themadman0980 commented 2 years ago

I'm having an issue with the new 'search' endpoint for resources. Every query I enter in the below format freezes my program during runtime. No errors or anything, it just simply sits there without making any apparent progress.

Comicvine.search(resource=simyan.comicvine.ComicvineResource.VOLUME,query='string here')

themadman0980 commented 2 years ago

One of the queries in question is 'Peter Parker: Spider-Man Annual 1999'. Is this just a problem caused by too many results, and if so do you have any suggestions on preventing this?

themadman0980 commented 2 years ago

Ok, yep. It ended up working with 4000+ results. Guess I just need to be careful how I use it.

themadman0980 commented 2 years ago

This seems to be a known issue with CV search API. A possible solution is to set a MAX_SEARCH_RESULTS parameter so that only the first 100 matches are returned?

https://github.com/cbanack/comic-vine-scraper/issues/460#issuecomment-451374161

Buried-In-Code commented 2 years ago

limit: The number of results to display per page. This value defaults to 10 and can not exceed this number. --Comicvine API docs

Currently there are 10 results per page being returned, according to the docs this was the maximum you could return on a search, so this will be why it took so long to return all the results as each request is ratelimited. In some testing I noticed that the limit can go above 10 so I'll update the script to include a new per-page limit, probably 100.

A possible solution is to set a MAX_SEARCH_RESULTS parameter so that only the first 100 matches are returned?

I'll have a think and test and see how best to implement this...

themadman0980 commented 2 years ago

Thanks for investigating this. To be honest, the max search results field would be useful in all CV endpoints. Generally the results I'm after exist in the first 100 results, so I could happily limit all of my searches to the first 100 only.