serpapi / public-roadmap

Public Roadmap for SerpApi, LLC (https://serpapi.com)
45 stars 3 forks source link

[Google Search API] `total_results` returned but not displayed in HTML #1617

Closed marm123 closed 1 month ago

marm123 commented 1 month ago

One of our Enterprise Customers reported that they are getting an inaccurate number of total_results returned. This is likely due to how Google serves results and the fact they are showing a different number of results returned that what we can actually get from that search query, but the information is missing on the HTML file:

image

It's especially weird that Google usually showed millions or thousands of results found (like in the example above), but the customer only got information about 68 total_results, which was still inaccurate.

Customer's search:

image

Playground | Inspect | Customer's Inspect Intercom

aciddjus commented 1 month ago

The results are still on the page, but Google has started hiding them by default.

If you inspect the page, search for the HTML item with classes pZvJc jf0C1c, and remove the display: none; CSS property, results will show up on the page.

Also, I'm getting a similarly low number of results for the customer's query on live Google search, so I don't see the issue with the number being incorrect.

image

marm123 commented 1 month ago

Thanks @aciddjus, that makes sense.

The odd thing about the query from the customer was that it showed 68 total_results when there were 15 in fact. Usually, Google showed millions or thousands of results on the first page, and 68 looked oddly specific and weird when there were no 68 results, but 15 only. But overall I agree that while this is kinda odd, it's just Google not showing the correct number of results as usual.

Since this is not an issue with our parser but rather Google updating the way they show (or in this case not show) the number of total_results, I'm closing this thread.

aciddjus commented 1 month ago

Thanks, @marm123. We can still fix hidden results in the HTML and show them by default. I'll open a quick PR.

The odd thing about the query from the customer was that it showed 68 total_results when there were 15 in fact. Usually, Google showed millions or thousands of results on the first page, and 68 looked oddly specific and weird when there were no 68 results, but 15 only. But overall I agree that while this is kinda odd, it's just Google not showing the correct number of results as usual.

Yeah, it's indeed unusual, but it is far from the first time that Google has served an incorrect number of results.

aciddjus commented 1 month ago

This issue has been resolved.

We are now showing total_results and time_taken_displayed in HTML.

image


Playground