Scheduled process to automatically fetch GA data via terminal/command line

[TomLin]

Hello, Sorry for bothering you, but I run into a problem when I was setting a scheduled task to fetch GA data via mac terminal or windows command line. The authentication process seems only working in R GUI or RStudio (for example, even if I run source(myRscript.r), it works.) But when i run the script in mac terminal, the response said: --Error: oauth_listener() needs an interactive environment. --Execution halted

Or if I run it in the windows command line, the response said: --Error: interactive() is not TRUE --Execution halted

I tried to follow up the instructions in the PDF document of auto-authentication. (In ga_auth() section). But I don't fully understand the logics behind it and also face the same problems as discussed in issue #41 Auto-authentication JSON file not working / Account list parsing fail.

Either I wrote the following codes: --Sys.setenv(GA_AUTH_FILE = "/Users/TomLin/myAuth.json")

--googleAuthR::gar_auth_service("/Users/TomLin/myauth.json")

I think maybe it's not the same issue as what I discussed here. But I am not sure.

Besides, I am not sure if googleAnalyticsR or any other packages allows for running R script in scheduled mode, because I tried to use RGA, but it fails too.

Thanks again for your help, I am really appreciated.

best wishes,

Tom

[MarkEdmondson1234]

Thanks for writing.

I think for both packages (RGA and this one), the authentication is somehow not seeing the authentication file. You can schedule scripts otherwise, we do it all the time :)

Its probably because the scheduler method you are using is executing the script in a different folder than what you expect. Perhaps try writing out some getwd()to any logs.

If using the service style of authentication (via the json file) you must also include the email address associated with that JSON file (service email) as a user to your GA account you are looking to pull data from.

Perhaps if you share what bash script you are using to try and schedule, that may help. I would also suggest using the cronR (OSX, Linux) or taskScheduleR to set up your schedule as they have helpful interfaces to help syntax mistakes.

[TomLin]

Hello Mark,

Thanks for your prompt answer. And yes, when I specify setwd() explicitly in the beginning of r script, it successfully loads in the saved auth token and doesn't jump into the interactive mode (namely pumping up the browser) anymore.

But one last question, pardon me, is the saved token permanently valid or somehow in the future, the authentication would require me to interact with browser again.

Thanks again.

[MarkEdmondson1234]

Great - ideally setwd() shouldn't be used in scripts as it makes it only work on your machine, but if thats the case you can worry about that later :)

The JSON file will work until you revoke it in the GCP console, the .httr-oauth file refreshes each time its used so as long as that file is there, it should work indefinitely.

[ErezLo]

Hi

I am sorry if its out of place but i am having kind of the same issue and I dont quite understand how you all managed to solve it :)

I am trying to use taskscheduleR to run API calls to ga.

lets assume this is the simple script i am trying to run

library(tidyverse)
library(googleAnalyticsR)
ga_auth()
account_list <- ga_account_list()
meta <- google_analytics_meta()
desktop_id <- 1234
as.Date(this_year_start <- "2017-07-01")
as.Date(this_year_end <- "2017-07-31")

desktop_events <- google_analytics_4(desktop_id,date_range = c(this_year_start, this_year_end),
                           metrics = c("totalEvents"),
                           dimensions = c("eventCategory", "eventAction"),
                           anti_sample = TRUE,
                           dim_filters = fc_desktop_events,
                           max=-1)

using tasksecedule plugin or even inline script like this using setwd (and even without it) its just not working

library(taskscheduleR)
setwd("C:/Users/Erez/Documents")
myscript <- system.file("extdata", "test.R", package = "taskscheduleR")
taskscheduler_create(taskname = "myfancyscript", rscript = myscript, schedule = "ONCE", starttime = format(Sys.time() + 62, "%H:%M"))

I am always getting oauth_listener() needs an interactive environment

would love to understand how you solved it.

Thank you in advance

[MarkEdmondson1234]

If you get oauth_listener() that 99% of time means your script is not finding the cache file, and is looking for interactive authentication again - it can't find either the .httr-oauth created the first time you authenticated or your JSON file. This often happens when you schedule scripts as the script is running from a different place than you expect. I would add messages that output the working directory of the script and compare to you expectation, or supply a full file path to the authentication file.

Don't use setwd() in scripts, it will cause more problems than fix.

[ErezLo]

Thanks Mark.

Can you guide me or send me to a tutorial on how to make the message or set the path to the .httr-oauth file ?

Is it supposed to be like this ?

Sys.setenv(GA_AUTH_FILE = "/Users/Erez/Documents/")

Thank you for your time

[MarkEdmondson1234]

Nearly, like this:

Sys.setenv(GA_AUTH_FILE = "/Users/Erez/Documents/.httr-oauth")

[ErezLo]

Thanks After running this command it seems like something is not working anymore in the package

Default Google Project for googleAnalyticsR is now set. This is shared with all googleAnalyticsR users. If making a lot of API calls, please: 1) create your own Google Project at https://console.developers.google.com 2) Activate the Google Analytics Reporting API 3) set options(googleAuthR.client_id) and options(googleAuthR.client_secret) 4) Reload the package. Error: oauth_listener() needs an interactive environment. Execution halted

just to make clear, all of these are steps I did in the past.

[ErezLo]

I think the interactive environment is fixed (?) but now I am getting a new error.

Default Google Project for googleAnalyticsR is now set. This is shared with all googleAnalyticsR users. If making a lot of API calls, please: 1) create your own Google Project at https://console.developers.google.com 2) Activate the Google Analytics Reporting API 3) set options(googleAuthR.client_id) and options(googleAuthR.client_secret) 4) Reload the package.

• Error: Failed to create local cache ('.httr-oauth') Execution halted

library(taskscheduleR)
Sys.setenv(GA_AUTH_FILE = "/Users/Erez/Documents/.httr-oauth")
myscript <- system.file("extdata", "test.R", package = "taskscheduleR")
taskscheduler_create(taskname = "myfancyscripttest1122111", rscript = myscript, schedule = "ONCE", starttime = format(Sys.time() + 62, "%H:%M"))

this is test.R

library(tidyverse)
library(googleAnalyticsR)
ga_auth()
desktop_id <- 1234
as.Date(this_year_start <- "2017-07-01")
as.Date(this_year_end <- "2017-07-31")
as.Date(last_year_start <- "2016-07-01")
as.Date(last_year_end <- "2016-07-31")

## DESKTOP REPORTING

adesktop_hostnames <- google_analytics(desktop_id,start=this_year_start, end=this_year_end,
                                       metrics = c("sessions"),
                                       dimensions = c("hostname"),
                                       max_results = 999999)

Thank you (again...)

[MarkEdmondson1234]

I think this error is Windows not letting you write to the same folder as where your script is running from (insufficient permissions?), which is related to your first error. I'm not familiar with Windows Task Manager, but can you somehow choose where the script it run? In Linux its "change directory" e.g. cd my/directory && runScript.R

[MarkEdmondson1234]

Try this:

library(tidyverse)
library(googleAnalyticsR)

googleAuthR::gar_auth(Sys.getenv("GA_AUTH_FILE"))
desktop_id <- 1234
as.Date(this_year_start <- "2017-07-01")
as.Date(this_year_end <- "2017-07-31")
as.Date(last_year_start <- "2016-07-01")
as.Date(last_year_end <- "2016-07-31")

## DESKTOP REPORTING

adesktop_hostnames <- google_analytics(desktop_id,start=this_year_start, end=this_year_end,
                                       metrics = c("sessions"),
                                       dimensions = c("hostname"),
                                       max_results = 999999)

And make sure you are setting the enviornment in your home directory (e.g. in RStudio, click Home in file explorer, create an .Renviron file and put this in it:

GA_AUTH_FILE = "/Users/Erez/Documents/.httr-oauth"

[ErezLo]

just to clarify, when executing my test.R from rstudio it runs and working I made the change to the test.R file and rescheduled the task (running from Rstudio it also works)

a new error :)

Default Google Project for googleAnalyticsR is now set. This is shared with all googleAnalyticsR users. If making a lot of API calls, please: 1) create your own Google Project at https://console.developers.google.com 2) Activate the Google Analytics Reporting API 3) set options(googleAuthR.client_id) and options(googleAuthR.client_secret) 4) Reload the package. Error in read_cache_token(token_path = token) : Cannot read token from alleged .rds file: Calls: -> read_cache_token Execution halted

you think its still has something to do with writing permissions ?

[MarkEdmondson1234]

I see you are on Windows, whoops :)

Get the right file path to put in GA_AUTH_FILE - I guess its something like "C://Documents/.httr-oauth".

You must be able to run file.exists(Sys.getenv("GA_AUTH_FILE")) so that it is TRUE before the script will work. Restart the R session in-between updating the .Renviron file between tests.

[ErezLo]

Hey

Yes I am on windows.

I changed it in the .Reviron file GA_AUTH_FILE = "C://Users/Erez/Documents/.httr-oauth" Restarted R I a getting TRUE for the file.exists(Sys.getenv("GA_AUTH_FILE")) but still and error after scheduling.

2017-09-05 15:00:31> Default Google Project for googleAnalyticsR is now set.  This is shared with all googleAnalyticsR users. 
 If making a lot of API calls, please: 
 1) create your own Google Project at https://console.developers.google.com 
 2) Activate the Google Analytics Reporting API 
 3) set options(googleAuthR.client_id) and options(googleAuthR.client_secret) 
 4) Reload the package.
Error in read_cache_token(token_path = token) : 
  Cannot read token from alleged .rds file:
Calls: <Anonymous> -> read_cache_token
Execution halted

Thanks

[MarkEdmondson1234]

Hmm. Can you try deleting the token at C://Users/Erez/Documents/.httr-oauth and trying again? Also perhaps try with a fresh session, and installing the github versions of googleAuthR and googleAnalyticsR.

[ErezLo]

Tried both. I even reinstalled R, Rstudio and all the packages

still getting

Error in read_cache_token(token_path = token) : 
  Cannot read token from alleged .rds file:
Calls: <Anonymous> -> read_cache_token
Execution halted

I am not sure what else can I do :| I ## really appreciate your help but I dont want to waste anymore of your time.

Thank you so much for your help. I will also thank you for the guidance and for all you are giving to this community

[MarkEdmondson1234]

Ok I'm sorry its not working for you, its weird. I can only think that the scheduler does not have permission to read the authentication file for some reason, but I don't know how that works on Windows.

[ErezLo]

Hi Mark,

Hope its okay, I have another question. I've tried the httr gitgub and I found out this.

It sounds like you are mixing two workflows for making a cached token available to a script.

.httr-oauth is httr's destination for automatically cached tokens and it has the potential to store multiple tokens. They are in a key-value store where the key is formed by the combination of endpoint, app, and scopes.

In googlesheets and, I'm guessing, googleAuthR, we offer an alternative workflow where we save exactly one token to a descriptively named .rds file. The vignette you link to has code for storing exactly one token in this way:

library(googlesheets) token <- gs_auth(cache = FALSE) saveRDS(token, file = "googlesheets_token.rds") It sounds like you are placing an .httr-oauth cache file somewhere (option 1) then are providing it to a function that expects a single token stored in an .rds file (option 2). I.e. you're mixing the two.

Is it possible to use the same function withing googleAuthR ?

I've tried this with not luck

> token <- ga_auth(cache=FALSE) nor token <- gar_auth(cache=FALSE) doesnt seem to work.

ga_auth(cache = FALSE) : unused argument (cache = FALSE)
gar_auth(cache = FALSE) : could not find function "gar_auth

Would love to hear your input.

Thank you

[MarkEdmondson1234]

ga_auth doesn't support supply a filepath to an authentication file, but googleAuthR::gar_auth() does, which is what ga_auth() wraps. See ?googleAuthR::gar_auth - if you have multiple tokens saved to the .httr-auth you can try:

token <- readRDS("your-token.rds")
googleAuthR:gar_auth(token[[1]])

the [[1]] will ensure its only one token used if you have multiple save in the file.

You may be running in to trouble if the token has not been deleted before getting new credentials.

[ErezLo]

Thanks. I think I am missing where and how I am getting the token.rds file.

I dont see this file in my working directory or anywhere else. the only place I recall having it was when I used "rga" package

Thanks

[MarkEdmondson1234]

You can either set the option("googleAuthR.httr_oauth_cache") to your file location, or use saveRDSdirectly with your token e.g.

token <- ga_auth()
saveRDS(token, file = "your-token.rds")

That is what should be happening anyway though - the .httr-oauth file is of rds format.

[TomLin]

Hello Mark,

I just want to come by and say thank you. Even though I am nobody, but I hope you know your works and sharing really makes an influence for someone around the world. I am very grateful to that.

Hope everything goes well with you. And wish one day I can meet you in person to pay my tribute.

best, Tom

On 11 Sep 2017, at 1:04 AM, Mark notifications@github.com<mailto:notifications@github.com> wrote:

You can either set the option("googleAuthR.httr_oauth_cache") to your file location, or use saveRDSdirectly with your token e.g.

token <- ga_auth() saveRDS(token, file = "your-token.rds")

That is what should be happening anyway though - the .httr-oauth file is of rds format.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/MarkEdmondson1234/googleAnalyticsR/issues/92#issuecomment-328356468, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AOoTpBBoIh2Plt6XbfhxQrMShpIgTSN2ks5shBaegaJpZM4OZYqy.

[MarkEdmondson1234]

Thanks @TomLin - its comments like this that keep me contributing. And as I read to my daughter, everybody is a somebody. :)