serpapi / public-roadmap

Public Roadmap for SerpApi, LLC (https://serpapi.com)
50 stars 3 forks source link

[Google Trends API] Searches Returning Incorrect Results #887

Open schaferyan opened 1 year ago

schaferyan commented 1 year ago

A customer reported Google Trends searches are sometimes returning inaccurate results. I haven't been able to replicate it in the playground, but they shared several "Interest over Time" searches that contain 0 values. I don't see a specific pattern in the dates used.

Screenshot 2023-05-23 at 9 40 30 AM

Search Inspector: 1 | 2 | 3 | 4 Intercom

schaferyan commented 1 year ago

Another customer reported this:

Intercom

schaferyan commented 1 year ago

Another customer asked about the status of this:

Intercom

schaferyan commented 1 year ago

Another customer followed up on this:

Intercom

schaferyan commented 1 year ago

I had thought this might have been on Google's side and could be already resolved. However I was able to recreate it in the Playground, so it is still a problem:

Playground

schaferyan commented 1 year ago

Another customer reported this:

Intercom

schaferyan commented 1 year ago

Another customer reported this:

Intercom

jbnitorum commented 1 year ago

Is this getting any attention? Seems like it just keeps getting reported without any resolution.

schaferyan commented 1 year ago

Moved this to urgent because of the high number of customers who have inquired about it.

However it's possible there won't be a fix for this. Pytrends experienced the same issue and concluded it was due to factors beyond their control:

https://github.com/GeneralMills/pytrends/issues/581

schaferyan commented 1 year ago

User shared a file with details of over 3,000 failed searches in case it is helpful for investigating:

Intercom

nicktba commented 1 year ago

is there any update at all on this?

Rendering the trends feature useless at this point.

Appreciate the hard work!

jbnitorum commented 1 year ago

This is now completely inoperable. Its been two months since I flagged and there has been no meaningful update.

hartator commented 1 year ago

This is now completely inoperable. Its been two months since I flagged and there has been no meaningful update.

Sorry about that. There are unfortunately several technical issues both our side and Google side at the moment with Google Trends data. We are working on it though. And hopefully we will have fixes soon.

marm123 commented 1 year ago

Another customer reported the issue:

Intercom

ilyazub commented 1 year ago

Data from Google Trends and SerpApi match and contain only positive values.

Google Trends SerpApi
image image
https://trends.google.com/trends/explore?date=2023-03-04T15%202023-03-05T15&geo=US&q=nissan&tz=420 https://serpapi.com/playground?engine=google_trends&q=nissan&geo=US&tz=420&date=2023-03-04T15+2023-03-05T15
aliayar commented 10 months ago

Another one reported here via email.

trends_missing

Inspect | Playground | Intercom

martin-serpapi commented 10 months ago

Another customer reported the issue.

image

Playground | Inspect

image

Google Trends

Intercom

martin-serpapi commented 10 months ago

Another customer reported the issue.

image

Playground | Inspect

image

Google Trends

Intercom

martin-serpapi commented 10 months ago

Another customer reported the issue:

Intercom

schaferyan commented 10 months ago

Another customer reported this:

Intercom

martin-serpapi commented 10 months ago

Another customer reported this:

Intercom

aliayar commented 9 months ago

Another customer reported this.

Intercom

ilyazub commented 9 months ago

Unfortunately, the Interest Over Time values in the response differ when we use a proxy, even with the exactly same URL and cookies as in the real browser. Without a proxy, values in the response are the same.

google_trends_sakura_curl_ruby_same_no_proxy_62_tz_300

Additionally, the widget URLs we use in the code are also different.

URL we used:

https://trends.google.com/trends/api/widgetdata/multiline?tz=300&req=%7B%22time%22%3A%222023-12-01T11%5C%5C%3A26%5C%5C%3A59+2023-12-08T11%5C%5C%3A26%5C%5C%3A59%22%2C%22resolution%22%3A%22HOUR%22%2C%22locale%22%3A%22en-US%22%2C%22comparisonItem%22%3A%5B%7B%22geo%22%3A%7B%7D%2C%22complexKeywordsRestriction%22%3A%7B%22keyword%22%3A%5B%7B%22type%22%3A%22BROAD%22%2C%22value%22%3A%22sakura%22%7D%5D%7D%7D%5D%2C%22requestOptions%22%3A%7B%22property%22%3A%22%22%2C%22backend%22%3A%22CM%22%2C%22category%22%3A0%7D%2C%22userConfig%22%3A%7B%22userType%22%3A%22USER_TYPE_SCRAPER%22%7D%7D&token=APP6_UEAAAAAZXRPpFCno2qIeTJuya_mkdccGxdScUHH

Google Trends UI:

https://trends.google.com/trends/api/widgetdata/multiline?hl=en-US&tz=300&req=%7B%22time%22:%222023-12-01T11%5C%5C:26%5C%5C:59+2023-12-08T11%5C%5C:26%5C%5C:59%22,%22resolution%22:%22HOUR%22,%22locale%22:%22en-US%22,%22comparisonItem%22:%5B%7B%22geo%22:%7B%7D,%22complexKeywordsRestriction%22:%7B%22keyword%22:%5B%7B%22type%22:%22BROAD%22,%22value%22:%22sakura%22%7D%5D%7D%7D%5D,%22requestOptions%22:%7B%22property%22:%22%22,%22backend%22:%22CM%22,%22category%22:0%7D,%22userConfig%22:%7B%22userType%22:%22USER_TYPE_LEGIT_USER%22%7D%7D&token=APP6_UEAAAAAZXRPA7alxgSor3RXKAUT0IGPTaWyBiB6&tz=300
martin-serpapi commented 9 months ago

Another customer reported this:

Intercom

nicktba commented 9 months ago

Any update on this?

nicktba commented 9 months ago
Screenshot 2023-12-18 at 12 42 51 AM
marm123 commented 9 months ago

Hi @nicktba. Unfortunately, we don't have a proper solution to solve the issue yet. Our Engineers are trying to find the solution, but the issue seems more complex than anticipated.

Meanwhile, another customer reported the same issue.

Intercom

edmarsj commented 8 months ago

Any updates on this issue?

schaferyan commented 8 months ago

Hi @edmarsj, unfortunately, we don't have an update on this issue at this time. Any fixes, workarounds, or decisions regarding the issue will be posted here. If you've contacted us via email or chat, we will also reach out to you there.

Thanks for your continued patience!

martin-serpapi commented 7 months ago

Another customer reported that when searching for "Compared Breakdown By Region" (data_type=GEO_MAP), the results are different than in an actual search in Google Trends.

image

Playground | Inspect

image

Google Trends Search

Intercom

marm123 commented 7 months ago

Another user reported the issue:

Intercom

schaferyan commented 6 months ago

Potential Customer followed up on this:

Intercom