serpapi / public-roadmap

Public Roadmap for SerpApi, LLC (https://serpapi.com)
54 stars 5 forks source link

[Google Search API] `related_searches` not returned on Mobile version #309

Closed marm123 closed 1 year ago

marm123 commented 2 years ago

There is no related_searches key returned while scraping Google Search on mobile.

image

Playground | Inspect

ilyazub commented 2 years ago

These Related searches are expected to be extracted into inline_people_also_search_for or related_search_boxes previously.

related_searches are ellipses with keywords at the bottom of the SERP.

image

Edit: Sorry. I've double-checked. Some pages contain both similarly-looking People Also Search For and Related Searches blocks. This is confusing. Search ID: 6347c4c0359e994965f10fbe.

People Also Search For Related Searches
image image
marm123 commented 1 year ago

Opening this thread as the issue back once again:

image

Playground | Inspect

marm123 commented 1 year ago

"People also search for" is also missing from the JSON response. It's a bit confusing with the naming, as Illia pointed out. I'm unsure if this is extracted in the same block, but if not, I can create a separate issue. LMK

image

Playground | Inspect

schaferyan commented 1 year ago

I'm reopening this thread as a customer just reported having this issue again. I was unable to replicate it in our Playground, however. Also it may be subtly different - feel free to move to a new thread if necessary. It's hard for me to see what's going on without being able to read Japanese, but in the example in the search_inspector it looks like we are only scraping query and link from "People also Ask" and not any of the other keys we usually parse..

image

search inspector

aciddjus commented 1 year ago

I'm reopening this thread as a customer just reported having this issue again. I was unable to replicate it in our Playground, however. Also it may be subtly different - feel free to move to a new thread if necessary. It's hard for me to see what's going on without being able to read Japanese, but in the example in the search_inspector it looks like we are only scraping query and link from "People also Ask" and not any of the other keys we usually parse..

image

search inspector

Not 100% sure I understand the problem. There is nothing here except query and link, and we are extracting it.

image
marm123 commented 1 year ago

@aciddjus I believe the confusion comes from language differences in API results. The issue customer has is that the "People also ask" is extracted together with related searches, while in fact, it's not "People also ask" but "People also search for". Below is the conversation with a customer about their issue:

Intercom conversation

I'm closing this one as the issue is more if we should extract "People also search for" together with "Related Searches" in the same related_searches block of the JSON response or not, and this thread is about something else.

aciddjus commented 1 year ago

@marm123 Got it 👍

Yes it's super confusing.