[Google AI Overview API] paragraphs in AI overviews are sometimes empty

[sonika-serpapi]

A customer reported that they are not seeing paragraphs populated in some of their Google Search AI overviews response.

I was able to replicate this on my end with a different search: https://serpapi.com/playground?q=is+coffee+good+for+health&location=Austin%2C+Texas%2C+United+States&gl=us&hl=en&google_domain=google.com&newPara=chips+as_sitesearch+highlight+stick+lr+as_qdr+votes+site+safe+google_domain+si+form+filter+as_epq+client+ie+oe+pws+ibp+source+rlz+ijn+interval

They also mentioned a couple of requests to get AI overviews return as an error (See search inspect 2):

I was not able to replicate this on my end. I do the see the paragraphs and a full response returned when I retried the queries. (See search inspect 3 and 4)

Internal links: Front | Search Inspect 1 | Search Inspect 2 | Search Inspect 3 | Search Inspect 4

[schaferyan]

Thank you @sonika-serpapi. I'm not sure about the missing paragraphs, but my guess is that the 404 error response may be a result of the page_token expiring.

Can you ask the customer whether they completed the search immediately after getting the token? Or get the search ID of the search where they obtained the page token (as we could then compare search creation times)?

[sonika-serpapi]

@schaferyan this is the information I am able to see from the inspect: Details for the initial search [has AI overviews token]: "created_at": "2024-11-07 20:05:28 UTC", "processed_at": "2024-11-07 20:05:28 UTC" Inspect

Details for the AI overviews result: "created_at": "2024-11-07 20:05:30 UTC", "processed_at": "2024-11-07 20:05:30 UTC" Inspect

[migsterrrrr]

@schaferyan can confirm I have created the request immediately after the token was provided.

I'm getting quite a few empty snippets recently - a few examples below

https://serpapi.com/searches/6730991dd11a89ddbf90cdda/inspect https://serpapi.com/searches/672e4ead213c8744f39af081/inspect https://serpapi.com/searches/672e4e001397ba74b13832b6/inspect https://serpapi.com/searches/67309921b28e6b954d420392/inspect

Happy to provide more if helpful

[hilmanski]

Thank you very much, @migsterrrrr , for the additional samples. If possible, can you please also share the original search of the Google Search that retrieved this AI page token?

We'll continue investigating this. Thank you very much for your patience.

[migsterrrrr]

Hey @hilmanski

Here's the last few searches that produced an empty AI response despite a token

1. this search: https://serpapi.com/searches/44b26cdd4a9233e0/6731db1cf9338866e6c25dfa.json

"search_parameters": {
    "engine": "google",
    "q": "what is trygroundswell.com?",
    "location_requested": "Austin, Texas, United States",
    "location_used": "Austin,Texas,United States",
    "google_domain": "google.com",
    "hl": "en",
    "gl": "us",
    "device": "desktop"

returned https://serpapi.com/searches/6731db1dcf32e04825af98d0/inspect

2. this search: https://serpapi.com/searches/70a0bd24555bf7fb/6731708b922cceff4ff0f624.json

"search_parameters": {
    "engine": "google",
    "q": "does goto.com integrate with Linkedin?",
    "location_requested": "Austin, Texas, United States",
    "location_used": "Austin,Texas,United States",
    "google_domain": "google.com",
    "hl": "en",
    "gl": "us",
    "device": "desktop"

returned https://serpapi.com/searches/6731708c74f0a49dcc82bc20/inspect

3. this search https://serpapi.com/searches/7a973743f602c442/67316874131502ce6470629f.json

"search_parameters": {
    "engine": "google",
    "q": "does goldcast.io integrate with Linkedin?",
    "location_requested": "Austin, Texas, United States",
    "location_used": "Austin,Texas,United States",
    "google_domain": "google.com",
    "hl": "en",
    "gl": "us",
    "device": "desktop"
  }

returned https://serpapi.com/searches/673168768b84c4e0fd8790e7/inspect

Do you have access to my logs? Basically any 400 from google you'd see the request. Otherwise, no issue can send you more examples across. Or build something to log this on my end.

🙇‍♂️

[schaferyan]

@migsterrrrr thanks for providing these details! This will be very helpful for our investigation.

Do you have access to my logs? Basically any 400 from google you'd see the request. Otherwise, no issue can send you more examples across. Or build something to log this on my end.

Feel free to share other examples if you have them, but this should be enough to proceed.

[zyc9012]

We've released a fix for empty response (Google 400 page). cc @migsterrrrr

[migsterrrrr]

AWESOME work everyone

wooho last 10 requests I have done all returned the snippet! 💪👏

[sonika-serpapi]

You're very welcome, @migsterrrrr. Thank you @zyc9012 for working on this.