related_queries NULL, data is not loaded

[pricerices]

hi,

The scrapping process is working properly. But related_queries data is not loaded (NULL).

I check Google Trends website manually and it shows keywords as usual.

could you solve this or is it from google side?

thx

[PMassicotte]

Please provide a reproducible example.

[pricerices]

I run this:

gtrends(c("what","where","when","who","why"), geo = "", cat = "0", time = "now 7-d")

and as simple as:

gtrends("what")

[eddelbuettel]

What happens when you run the known working example from the README.md and documentation?

[pricerices]

I can't get related_queries data. This happened since 5 days ago.

[pricerices]

for more detail example, I run this:

allcat <- gtrends(c("what","where","when","who","why"), geo = "", cat = "0", time = "now 7-d")

allcatrq <- allcat$related_queries %>% filter(related_queries == "rising") %>% select(value)

and the error result in console:

Error in UseMethod("filter") : no applicable method for 'filter' applied to an object of class "NULL"

Experiencing this for the first time. Even though keywords/related_queries appear normally in google trends web.

[eddelbuettel]

I can confirm with a test query. We don't seem to be getting that data back even though the web interface has it for the same query. :-/

[PMassicotte]

I will take a look tomorrow. Likely something changed in they payload options.

[PMassicotte]

Just did a quick search and there are not much difference between us and Google Trends

[ins] r$>   waldo::compare(fromJSON(p1), fromJSON(p2), x_arg = "gtrendsR", y_arg = "web")
`gtrendsR$restriction$time`: "2024-08-18T14\\:05\\:57 2024-08-25T14\\:05\\:57"
`web$restriction$time`:      "2024-08-18T14\\:43\\:56 2024-08-25T14\\:43\\:56"

gtrendsR$restriction$complexKeywordsRestriction$keyword vs web$restriction$complexKeywordsRestriction$keyword
                                                               value
- gtrendsR$restriction$complexKeywordsRestriction$keyword[1, ]   NHL
+ web$restriction$complexKeywordsRestriction$keyword[1, ]        nhl

`gtrendsR$restriction$complexKeywordsRestriction$keyword$value`: "NHL"
`web$restriction$complexKeywordsRestriction$keyword$value`:      "nhl"

`gtrendsR$trendinessSettings$compareTime`: "2024-08-11T14\\:05\\:57 2024-08-18T14\\:05\\:57"
`web$trendinessSettings$compareTime`:      "2024-08-11T14\\:43\\:56 2024-08-18T14\\:43\\:56"

`gtrendsR$userConfig$userType`: "USER_TYPE_SCRAPER"   
`web$userConfig$userType`:      "USER_TYPE_LEGIT_USER"

[nicolacaravaggio]

I am following this thread since (I guess) it is connected to my problem. I am having trouble downloading data from GT through gtrendsR, results are always empty while when I am doing the exact same query with pytrends I am getting the results with no problem. I checked with some colleagues of mine and they have the same problem.

Just as an example, where I am having NULL as result:

library(gtrendsR)

gt_data <- gtrends(
  keyword = "Taylor Swift",         
  geo = "US",               
  time = "today 12-m"        
)

head(gt_data$interest_over_time)

[PMassicotte]

As previously said, this is not something easily fixable, as it appears that Google knows that we are not using the official web page and thus seems to block some content (https://github.com/PMassicotte/gtrendsR/issues/465#issuecomment-2308880707).