[Google Search API] Provide actual results number

[hilmanski]

Context: Right now, we're providing the total_search_results based on what Google provides. But this number doesn't represent the actual returned results.

VS the actual number

I think we can get the actual number by adding &start=990 in the query. Since Google limited their results up to 1000, and got the number from the bottom part of the page (see in the screenshot above)

When using 991 or above

Notes: we have to add an extra scrape to this particular page, &start=990. So, if it's possible, maybe we could provide this as a "separate API".

Example end of page
Intercom1 || Intercom2

[hartator]

As it's not possible to do without firing a new request for every search, we can't fix this unfortunately.

[hilmanski]

What if this is a separate API? Cost 1 credit as well, since some users need it. /total-google-serp-results-api

[hartator]

What if this is a separate API? Cost 1 credit as well, since some users need it. /total-google-serp-results-api

Yes, why not.

It might be overkill to have a full API just for start=990 hack. Maybe a blog post? We already have Justin that topped HN: https://serpapi.com/blog/googles-millions-of-search-results-are-not-being-served-in-the-later-pages-search-results/

We can have "How to get accurate number of results on Google" or something like that.

[hilmanski]

It might be overkill, but not if some customers really need it. Let's see if there is more demand for this in the future.

[hilmanski]

This could be an alternative when Google is not displaying the total_results on a search.

Related issue: https://github.com/serpapi/public-roadmap/issues/1313 Example case