PMassicotte / gtrendsR

R functions to perform and display Google Trends queries
352 stars 112 forks source link

Daily search data for a longer period #425

Closed prasadsourav closed 1 year ago

prasadsourav commented 2 years ago

I want to download daily interval data for the period January 2004 to January 2019 for a range of terms for the US only. I followed the this website to do the same https://medium.com/@bewerunge.franz/google-trends-how-to-acquire-daily-data-for-broad-time-frames-b6c6dfe200e6 But this is giving me error.

I have used the following code t <- c("all","2004-01-01 2004-01-31", "2004-02-01 2004-02-29", "2004-03-01 2004-03-31", "2004-04-01 2004-04-30", "2004-05-01 2004-05-31", "2004-06-01 2004-06-30", "2004-07-01 2004-07-31", "2004-08-01 2004-08-31", "2004-09-01 2004-09-30", "2004-10-01 2004-10-31", "2004-11-01 2004-11-30", "2004-12-01 2004-12-31", "2005-01-01 2005-01-31", "2005-02-01 2005-02-28", "2005-03-01 2005-03-31", "2005-04-01 2005-04-30", "2005-05-01 2005-05-31", "2005-06-01 2005-06-30", "2005-07-01 2005-07-31", "2005-08-01 2005-08-31", "2005-09-01 2005-09-30", "2005-10-01 2005-10-31", "2005-11-01 2005-11-30", "2005-12-01 2005-12-31", "2006-01-01 2006-01-31", "2006-02-01 2006-02-28", "2006-03-01 2006-03-31", "2006-04-01 2006-04-30", "2006-05-01 2006-05-31", "2006-06-01 2006-06-30", "2006-07-01 2006-07-31", "2006-08-01 2006-08-31", "2006-09-01 2006-09-30", "2006-10-01 2006-10-31", "2006-11-01 2006-11-30", "2006-12-01 2006-12-31", "2007-01-01 2007-01-31", "2007-02-01 2007-02-28", "2007-03-01 2007-03-31", "2007-04-01 2007-04-30", "2007-05-01 2007-05-31", "2007-06-01 2007-06-30", "2007-07-01 2007-07-31", "2007-08-01 2007-08-31", "2007-09-01 2007-09-30", "2007-10-01 2007-10-31", "2007-11-01 2007-11-30", "2007-12-01 2007-12-31", "2008-01-01 2008-01-31", "2008-02-01 2008-02-29", "2008-03-01 2008-03-31", "2008-04-01 2008-04-30", "2008-05-01 2008-05-31", "2008-06-01 2008-06-30", "2008-07-01 2008-07-31", "2008-08-01 2008-08-31", "2008-09-01 2008-09-30", "2008-10-01 2008-10-31", "2008-11-01 2008-11-30", "2008-12-01 2008-12-31", "2009-01-01 2009-01-31", "2009-02-01 2009-02-28", "2009-03-01 2009-03-31", "2009-04-01 2009-04-30", "2009-05-01 2009-05-31", "2009-06-01 2009-06-30", "2009-07-01 2009-07-31", "2009-08-01 2009-08-31", "2009-09-01 2009-09-30", "2009-10-01 2009-10-31", "2009-11-01 2009-11-30", "2009-12-01 2009-12-31", "2010-01-01 2010-01-31", "2010-02-01 2010-02-28", "2010-03-01 2010-03-31", "2010-04-01 2010-04-30", "2010-05-01 2010-05-31", "2010-06-01 2010-06-30", "2010-07-01 2010-07-31", "2010-08-01 2010-08-31", "2010-09-01 2010-09-30", "2010-10-01 2010-10-31", "2010-11-01 2010-11-30", "2010-12-01 2010-12-31", "2011-01-01 2011-01-31", "2011-02-01 2011-02-28", "2011-03-01 2011-03-31", "2011-04-01 2011-04-30", "2011-05-01 2011-05-31", "2011-06-01 2011-06-30", "2011-07-01 2011-07-31", "2011-08-01 2011-08-31", "2011-09-01 2011-09-30", "2011-10-01 2011-10-31", "2011-11-01 2011-11-30", "2011-12-01 2011-12-31", "2012-01-01 2012-01-31", "2012-02-01 2012-02-29", "2012-03-01 2012-03-31", "2012-04-01 2012-04-30", "2012-05-01 2012-05-31", "2012-06-01 2012-06-30", "2012-07-01 2012-07-31", "2012-08-01 2012-08-31", "2012-09-01 2012-09-30", "2012-10-01 2012-10-31", "2012-11-01 2012-11-30", "2012-12-01 2012-12-31", "2013-01-01 2013-01-31", "2013-02-01 2013-02-28", "2013-03-01 2013-03-31", "2013-04-01 2013-04-30", "2013-05-01 2013-05-31", "2013-06-01 2013-06-30", "2013-07-01 2013-07-31", "2013-08-01 2013-08-31", "2013-09-01 2013-09-30", "2013-10-01 2013-10-31", "2013-11-01 2013-11-30", "2013-12-01 2013-12-31", "2014-01-01 2014-01-31", "2014-02-01 2014-02-28", "2014-03-01 2014-03-31", "2014-04-01 2014-04-30", "2014-05-01 2014-05-31", "2014-06-01 2014-06-30", "2014-07-01 2014-07-31", "2014-08-01 2014-08-31", "2014-09-01 2014-09-30", "2014-10-01 2014-10-31", "2014-11-01 2014-11-30", "2014-12-01 2014-12-31", "2015-01-01 2015-01-31", "2015-02-01 2015-02-28", "2015-03-01 2015-03-31", "2015-04-01 2015-04-30", "2015-05-01 2015-05-31", "2015-06-01 2015-06-30", "2015-07-01 2015-07-31", "2015-08-01 2015-08-31", "2015-09-01 2015-09-30", "2015-10-01 2015-10-31", "2015-11-01 2015-11-30", "2015-12-01 2015-12-31", "2016-01-01 2016-01-31", "2016-02-01 2016-02-29", "2016-03-01 2016-03-31", "2016-04-01 2016-04-30", "2016-05-01 2016-05-31", "2016-06-01 2016-06-30", "2016-07-01 2016-07-31", "2016-08-01 2016-08-31", "2016-09-01 2016-09-30", "2016-10-01 2016-10-31", "2016-11-01 2016-11-30", "2016-12-01 2016-12-31", "2017-01-01 2017-01-31", "2017-02-01 2017-02-28", "2017-03-01 2017-03-31", "2017-04-01 2017-04-30", "2017-05-01 2017-05-31", "2017-06-01 2017-06-30", "2017-07-01 2017-07-31", "2017-08-01 2017-08-31", "2017-09-01 2017-09-30", "2017-10-01 2017-10-31", "2017-11-01 2017-11-30", "2017-12-01 2017-12-31", "2018-01-01 2018-01-31", "2018-02-01 2018-02-28", "2018-03-01 2018-03-31", "2018-04-01 2018-04-30", "2018-05-01 2018-05-31", "2018-06-01 2018-06-30", "2018-07-01 2018-07-31", "2018-08-01 2018-08-31", "2018-09-01 2018-09-30", "2018-10-01 2018-10-31", "2018-11-01 2018-11-30", "2018-12-01 2018-12-31", "2019-01-01 2019-01-31", "2019-02-01 2019-02-28", "2019-03-01 2019-03-31", "2019-04-01 2019-04-30", "2019-05-01 2019-05-31", "2019-06-01 2019-06-30", "2019-07-01 2019-07-31", "2019-08-01 2019-08-31", "2019-09-01 2019-09-30", "2019-10-01 2019-10-31", "2019-11-01 2019-11-30", "2019-12-01 2019-12-31")

GOLD_PRICES <- gtrends( keyword = (c("GOLD PRICES")), geo = (c("US")), time = t, gprop = c("web"), category = 0, hl = "en-US", compared_breakdown = FALSE, low_search_volume = FALSE, cookie_url = "http://trends.google.com/Cookies/NID", tz = 0, onlyInterest = FALSE )

How to solve this?

eddelbuettel commented 2 years ago

Please see some of the earlier discussions archived under closed issues in this repo. In a nutshell, we do not control the backend that Google offers. So if a query of yours does not return, it means your query did not suit their requirements. So you have to change your query. We make no judgement, and we are "merely" the messenger carrying your query to the backend, and in the case of a result bring it back to you.

prasadsourav commented 1 year ago

Dear @eddelbuettel

I hope you are well.

I know I disturbed you thrice to help me get daily google trends data for an extended period. Unintentionally, I disturbed you.

Well, I found a way to collect gtrends daily frequency data for multiple keywords (more than five) for an extended period (2004- till date).

Hope this helps.

The google trends data can be collected using two loops with the following code.

library(gtrendsR)

month <- c("2004-01-01 2004-03-31", "2004-04-01 2004-06-30", "2004-07-01 2004-09-30","2004-10-01 2004-12-31", "2005-01-01 2005-03-31", "2005-04-01 2005-06-30", "2005-07-01 2005-09-30", "2005-10-01 2005-12-31", "2006-01-01 2006-03-31", "2006-04-01 2006-06-30", "2006-07-01 2006-09-30", "2006-10-01 2006-12-31", "2007-01-01 2007-03-31", "2007-04-01 2007-06-30", "2007-07-01 2007-09-30", "2007-10-01 2007-12-31", "2008-01-01 2008-03-31", "2008-04-01 2008-06-30", "2008-07-01 2008-09-30", "2008-10-01 2008-12-31", "2009-01-01 2009-03-31", "2009-04-01 2009-06-30", "2009-07-01 2009-09-30", "2009-10-01 2009-12-31", "2010-01-01 2010-03-31", "2010-04-01 2010-06-30", "2010-07-01 2010-09-30", "2010-10-01 2010-12-31", "2011-01-01 2011-03-31", "2011-04-01 2011-06-30", "2011-07-01 2011-09-30", "2011-10-01 2011-12-31", "2012-01-01 2012-03-31", "2012-04-01 2012-06-30", "2012-07-01 2012-09-30", "2012-10-01 2012-12-31", "2013-01-01 2013-03-31", "2013-04-01 2013-06-30", "2013-07-01 2013-09-30", "2013-10-01 2013-12-31", "2014-01-01 2014-03-31", "2014-04-01 2014-06-30", "2014-07-01 2014-09-30", "2014-10-01 2014-12-31", "2015-01-01 2015-03-31", "2015-04-01 2015-06-30", "2015-07-01 2015-09-30", "2015-10-01 2015-12-31", "2016-01-01 2016-03-31", "2016-04-01 2016-06-30", "2016-07-01 2016-09-30", "2016-10-01 2016-12-31", "2017-01-01 2017-03-31", "2017-04-01 2017-06-30", "2017-07-01 2017-09-30", "2017-10-01 2017-12-31", "2018-01-01 2018-03-31", "2018-04-01 2018-06-30", "2018-07-01 2018-09-30", "2018-10-01 2018-12-31", "2019-01-01 2019-03-31", "2019-04-01 2019-06-30", "2019-07-01 2019-09-30", "2019-10-01 2019-12-31")

keywords <- c("hello", "sourav", "this", "side", "default", "benefits", "unemployed", "poverty", "social security office")

google_data = data.frame() (for (i in keywords) { for (j in month) { g <- gtrends(keyword = i, geo = c("US"), time = j, gprop = "web")[1] if (!is.null(t)) output <- data.frame(g) google_data <- rbind(google_data,output) } } )

Hope this helps.