Error in `tmp*`[[i]] : subscript out of bounds (2 ways authentication)

[jriou]

Hi, Thanks for what looks to be very useful. However, I can't manage to use the gtrends() function, even with the provided examples:

gconnect("mygoogleaccount ","*_") res <- gtrends(c("nhl", "nba", "nfl")) Error in `_tmp`[[i]] : subscript out of bounds

I'm sure this must be on my hand but I can't find the reason...

Best regards, Julien

sessionInfo() R version 3.2.2 (2015-08-14) Platform: x86_64-apple-darwin13.4.0 (64-bit) Running under: OS X 10.11.1 (El Capitan)

locale: [1] fr_FR.UTF-8/fr_FR.UTF-8/fr_FR.UTF-8/C/fr_FR.UTF-8/fr_FR.UTF-8

attached base packages: [1] stats graphics grDevices utils datasets methods base

other attached packages: [1] gtrendsR_1.3.0 RCurl_1.95-4.7 bitops_1.0-6

loaded via a namespace (and not attached): [1] Rcpp_0.12.2 lattice_0.20-33 zoo_1.7-12 digest_0.6.8 MASS_7.3-45 grid_3.2.2
[7] plyr_1.8.3 gtable_0.1.2 magrittr_1.5 scales_0.3.0 ggplot2_1.0.1 stringi_1.0-1
[13] reshape2_1.4.1 googleVis_0.5.10 proto_0.3-10 RColorBrewer_1.1-2 RJSONIO_1.3-0 tools_3.2.2
[19] stringr_1.0.0 munsell_0.4.2 colorspace_1.2-6

[PMassicotte]

Can you try with the dev version of the package?

[jriou]

Thanks for answering so quickly, I tried and it's the same thing...

library(devtools) dev_mode(on=T) installgithub("PMassicotte/gtrendsR") d> gconnect("**","*_",verbose=TRUE) Google login successful! d> res <- gtrends(c("nhl", "nba", "nfl")) Error in `tmp*`[[i]] : indice hors limites

[zmijunkie]

Same here ... but I haven't tried the dev version of the package

[DaRobert]

the same here, I'm using the dev version.

[eddelbuettel]

For what it is worth I am pretty serious about two-factor authentication for my main account, so I create a "spare" for this work. It uses just username and password, and that seems to work just fine.

Whenever something irregular happens, Google may send email to the email address associated with the username. Checking there may give a clue.

[eddelbuettel]

So I just pulled from GitHub and I also get

R> library(gtrendsR)
Google login failed!
R> 

When I revert to the released version things work:

edd@max:~$ install.r gtrendsR                   ## install.r is a helper from littler 
trying URL 'https://cran.rstudio.com/src/contrib/gtrendsR_1.3.0.tar.gz'
Content type 'application/x-gzip' length 107729 bytes (105 KB)
==================================================
downloaded 105 KB

* installing *source* package ‘gtrendsR’ ...
** package ‘gtrendsR’ successfully unpacked and MD5 sums checked
** R
** data
** inst
** preparing package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded
* DONE (gtrendsR)

The downloaded source packages are in
        ‘/tmp/downloaded_packages’
edd@max:~$ r -lgtrendsR -e'print(head(gtrends(c("NBA","NFL"))$trend))'
       start        end NBA NFL
1 2004-01-04 2004-01-10  24  35
2 2004-01-11 2004-01-17  25  34
3 2004-01-18 2004-01-24  27  33
4 2004-01-25 2004-01-31  25  24
5 2004-02-01 2004-02-07  24  35
6 2004-02-08 2004-02-14  31  19
edd@max:~$ 

[eddelbuettel]

Philippe,

It is the

grepl("SetSID", authenticatePage)

which is FALSE for me (on either page).

For me

grepl("HTTP/1.1 200 OK", authenticatePage2)

would work.

[PMassicotte]

I will have to test more. There is something I do not understand quite well at the moment :)

[MathieuDe]

If it may helps (same error here): library(gtrendsR) ch<-gconnect(usr="mymail",psw="mypwd") res <- gtrends(c("nhl", "nba", "nfl")) Error in *tmp*[[i]] :subscript out of bounds

However using gtrend https://trinkerrstuff.wordpress.com/2014/11/24/gtrendsr-package-to-explore-google-trending-for-field-dependent-terms

library(gtrend) out <- gtrend_scraper("mymail","mypwd",terms, geo="FR") Loading required package: GTrendsR

Attaching package: ‘GTrendsR’

The following objects are masked from ‘package:gtrendsR’:

gconnect, gtrends

Works fine

[eddelbuettel]

@MathieuDe Pointing to Tyler's package means nothing; it connects different. We have (in the simplest case) an overzealous rejection of correct connections (as I showed), but (in the worst case) also a lack of proper affirmation of a working connection.

@PMassicotte Just to think out of the box: how about if upon connection we actually run a (small, very paramterized) query in a try/catch and see that the result is good? If so we're good :smiley: but if not we signal error. That is more or less how configure tests if you can link against a library: by actually building a minimal program linked against the library. We could make that suppressable by an option too...

[PMassicotte]

Good idea @eddelbuettel ! I started to get something today. I ll test it more seriously tomorrow on different computers at job. If it fails we can try to implement your idea!

[eddelbuettel]

@PMassicotte : I asked for help over within rOpenSci and the always-very-resourceful @jennybc (who knows this stuff rather well from her work with Google Spreadsheets [as an R-accessible data store]) pointed to this googleAuthR package. In case we find nuttin' better to do ...

[jennybc]

Picking up on @eddelbuettel ... I don't know if the Google Trends API is compatible with googleAuthR. But if it is, I think it's a good idea for all of us wrapping Google APIs to standardize on an auth package and work together on it. It will also make life easier for the user.

[PMassicotte]

I'll have a look to it @jennybc ! Tx for the head up.

Day is over for me in DK, see you tomorrow 😎

[PMassicotte]

@eddelbuettel I think I found it! When you have time, can you look at the new branch I pushed? I now use real HTTP response to validate if connection is successful. I also use Google cookie URL to validate the connection. If it works I will finalize the code and send a PR.

Cheers!

[eddelbuettel]

That looks good and should work.

But the proof is in the pudding. I'd say do the PR. we let people test the dev version in master for a day or two and then ship as 1.3.1?

[PMassicotte]

Sounds good my friend @eddelbuettel ! Will do that today :)

[tokumotion]

Still got the same problem with new version. But I read something that @eddelbuettel said about Google sending emails everytime something goes weird. Everytime I try to use the package, I get an email. That's something wrong on my account's side or the package?

[eddelbuettel]

That is you. At least as was in my case.

[PMassicotte]

@tokumotion do you get a successful connection with the new version? Can you try with verbose = TRUE?

[tokumotion]

@PMassicotte yes, I get a succesful connection.

[eddelbuettel]

Yay. I smell am upcoming 1.3.1 release :smiley:

[PMassicotte]

hehe definitely!

[jriou]

Hi all, thank you for your time, I'll try the new version right away!

[Israel48]

ups! I have deactivated two-step verification and still get the same error

ch=gconnect(usr=usr, psw=psw, verbose=T) Google login successful!

res <- gtrends(c("nhl", "nba", "nfl")) Error in *tmp*[[i]] : subscript out of bounds

[eddelbuettel]

Try creating a new account for this. That seems to work for everybody else. Keep two-factor for your main Google/Gmail accounts. It is safer.

[PMassicotte]

Ideas:

https://support.google.com/a/answer/1032419?hl=en http://ximen.es/?p=653 http://stackoverflow.com/questions/28346515/implementing-2-step-authentication-with-google-clientapi-for-python https://www.codementor.io/php/tutorial/google-two-step-authentication-otp-generation

[Israel48]

I did as Dirk said and created a new account without two step authentication butstill can't make queries after logging in. Same subscript out of bounds mssg:

ch<-gconnect(usr=usr,psw=psw, verbose=T) Google login successful! trend<- gtrends("facebook") Error in *tmp*[[i]] : subscript out of bounds

[eddelbuettel]

Can you execute the steps in the gtrends() function one by one and see if you get any data, and at what point it fails? Also save the data you get (i.e. via saveRDS(obj, file="/tmp/data.rds")) would be good as you could then email us the interim object obj via the file data.rds.

[Israel48]

Hi, not sure if this is what you mean, but I run this on the console: ch<-gconnect(usr=usr,psw=psw, verbose=T) authenticatePage2 <- getURL("http://www.google.com", curl = ch) trendsURL <- "http://www.google.com/trends/trendsReport?" start_date = as.Date("2012-01-01") pp <- list(q = "facebook", cmpt = "q", content = 1, export = 1, date = paste(format(start_date, "%m/%Y"), res), geo = "world") resultsText <- getForm(trendsURL, .params = pp, curl = ch)

resultText returns this: "<div id=\"report\">\n <div class=\"errorTitle\">Se ha detectado un error.

[sebastien-foulle]

Hi,

for a French user, the message "Error in *tmp*[[i]] : subscript out of bounds" results from the expressions "schidx <- grep("Top searches", headers) ... schlist <- lapply(1:length(schidx)..." in gtrendsR:::.processResults because schidx is equal to integer(0).

It should be replaced in France by "schidx <- grep("Recherches les plus fréquentes", headers) ... schlist <- lapply(seq_along(schidx)..."

or for the whole world by "schidx <- c(7,8) ... schlist <- lapply(seq_along(schidx)..."

Similar changes should be done for all "grep(XXX, headers)" expressions in gtrendsR:::.processResults.

Best regards,

Sebastien

[tokumotion]

It works!

Thanks you for the new release!

[PMassicotte]

Merci @sebastien-foulle, c'est probablement un truc de language. On va y jetter un oeil.

Est-ce que tu pourrais m'envoyer en format RDS les objects resultsText et queryparams ?

[Israel48]

Installed the new release and it works perfectly! thank you, guys

[PMassicotte]

We need to verify what returns gconnect when done on 2-ways authentication account. I cant verify it since I do not have a cell phone (and proud of it!).

I think we are close to be done with the connection issue, just need to figure out how to split data when returning in non-english language. Maybe open a new issue for this.

[eddelbuettel]

I do have a cell phone and could test, especially if you send me code snippets. I guess gconnect and an rds of the return? Other calls?

A little pressed for time too though...

[sebastien-foulle]

Bonsoir Philippe,

voilà en pièce jointe un script R et les deux fichiers rds qu'il produit. J'ai rajouté dans le script les infos sur ma version de R et sur la session, pour info.

Cordialement Sébastien Foulle

2015-12-07 8:38 GMT+01:00 Philippe Massicotte notifications@github.com:

Merci @sebastien-foulle https://github.com/sebastien-foulle, c'est probablement un truc de language. On va y jetter un oeil.

— Reply to this email directly or view it on GitHub https://github.com/PMassicotte/gtrendsR/issues/31#issuecomment-162438265 .

[PMassicotte]

@sebastien-foulle, pourrais-tu directement m'envoyer tout ca à pmassicotte [at] hotmail.com ? Merci d'avance!

[andraderdz]

Yes, it is a language issue, i'm in mexico and google return the "resultsText" variable in spanish... How can I modify the gtrendsR package from mi computer, so I can work every time.... i do not expect to change my residential soon. I do not know much about creating R package... so I go to my directory I funny to hope for and R file to change.... how I can change it from the root? thanks for help!

[eddelbuettel]

This thread is not the place to teach you about R and package building. It would probably be best if you tried some of the many (free) resources on the Internet.