search

jarun / googler

:mag: Google from the terminal
GNU General Public License v3.0
6.11k stars 529 forks source link

Possible to get total number of results for google news?' #340

Closed onseau closed 4 years ago

onseau commented 4 years ago

the number typically tops out at a few hundred for a given term on google news, so would it be possible to set results to 100 per page and run loop to get to the last page and getting the total number of results?

jarun commented 4 years ago

Sorry, this has been discussed earlier. We do not support scripting as it's against google policies.

onseau commented 4 years ago

sorry i'm not a programmer. what is scripting? or specifically, what part of that should i try to avoid? i don't understand

onseau commented 4 years ago

reason i'm asking is because i was doing this manually, so if i can't do that, could i use python to find the number of pages at least? or get the number of results on the last page? etc., then i can estimate from there.

jarun commented 4 years ago

You shouldn't try to fetch mass results from google in loop.

There's an unsupported patch to get the total number of results: https://github.com/jarun/googler/issues/144#issuecomment-260716097

Beyond that you are on your own.

onseau commented 4 years ago

thank you!!!

onseau commented 4 years ago

ahh ok. problem is, it has to be done this way, because of how google calculates that number on the first page. it drops exponentially until you reach the last page, where the actual reported number is accurate. thank you anyway! i'll look for another way.