get_realtime fails first time but succeeds second time

ginoa commented 5 years ago

Context

I am using RGA::get_realtime from an openshift environment.

Issue observed

Usage of RGA::get_realtime results in the following error Error in gzfile(file, mode) : cannot open the connection In addition: Warning message: In gzfile(file, mode) : cannot open compressed file '.<XXXXX>-token.rds', probable reason 'Permission denied' (instead of <XXXXXX> there is my GA email address, I removed it from this github issue for safety concerns)

NOTE: running get_realtime twice in a row with the same parameters results in failure of the first call and success of the second call. Weird behavior.

Tests performed

I tested using RGA::get_realtime from my local machine, no problem there. -> Problem appears only on the openshift environment.
I checked whether the token is correctly read from the .rds file into R session. The token seems to be correctly read from file.
I isolated the problem down to the function RGA:::list_mgmt: calling RGA:::list_mgmt twice results in the same issue: the first call fails with the error above, the second call succeeds. Maybe the problem is further down in RGA:::get_data but I don't know
I checked with identical() whether calling RGA:::list_mgmt changes the token. It does not.

Calling RGA::get_realtime a first time wrapped in httr::with_verbose results in following (NOTE: I edited out sensitive information by substituting it with <XXXX>)

> RGA::get_realtime(profileId = profileId, metrics = metrics,
       dimensions = dimensions, filters = filters,
       token = token)
-> GET /oauth2/v1/tokeninfo?access_token=<XXXXXXXX> HTTP/1.1
-> Host: www.googleapis.com
-> User-Agent: libcurl/7.52.1 r-curl/3.3 httr/1.4.0
-> Accept-Encoding: gzip, deflate
-> Accept: application/json
->
<- HTTP/2 400
<- pragma: no-cache
<- cache-control: no-cache, no-store, max-age=0, must-revalidate
<- date: Fri, 12 Apr 2019 08:03:35 GMT
<- expires: Mon, 01 Jan 1990 00:00:00 GMT
<- content-type: application/json; charset=UTF-8
<- vary: Origin
<- vary: X-Origin
<- vary: Referer
<- content-encoding: gzip
<- server: ESF
<- content-length: 78
<- x-xss-protection: 0
<- x-frame-options: SAMEORIGIN
<- x-content-type-options: nosniff
<- alt-svc: quic=":443"; ma=2592000; v="46,44,43,39"
<-
-> POST /o/oauth2/token HTTP/1.1
-> Host: accounts.google.com
-> User-Agent: libcurl/7.52.1 r-curl/3.3 httr/1.4.0
-> Accept-Encoding: gzip, deflate
-> Accept: application/json, text/xml, application/xml, */*
-> Content-Type: application/x-www-form-urlencoded
-> Content-Length: 208
->
>> refresh_token=<XXXXXXX>

<- HTTP/2 200
<- content-type: application/json; charset=utf-8
<- vary: Origin
<- vary: X-Origin
<- vary: Referer
<- content-encoding: gzip
<- date: Fri, 12 Apr 2019 08:03:35 GMT
<- server: ESF
<- cache-control: private
<- content-length: 250
<- x-xss-protection: 0
<- x-frame-options: SAMEORIGIN
<- x-content-type-options: nosniff
<- alt-svc: quic=":443"; ma=2592000; v="46,44,43,39"
<-
Error in gzfile(file, mode) : cannot open the connection
In addition: Warning message:
In gzfile(file, mode) :
  cannot open compressed file '.<XXXXXXX>-token.rds', probable reason 'Permission denied'

> traceback()
13: gzfile(file, mode)
12: saveRDS(tokens, cache_path)
11: cache_token(self, path)
10: self$cache()
9: token$refresh()
8: validate_token(token)
7: api_request(get_url(path, query), token)
6: get_data(path, query, token)
5: get_report("data/realtime", query, token)
4: RGA::get_realtime(profileId = profileId, metrics = metrics,
       dimensions = dimensions, filters = filters,
       token = token)
3: force(expr)
2: with_config(verbose(...), expr)
1: httr::with_verbose(RGA::get_realtime(profileId = profileId, metrics = metrics,
       dimensions = dimensions, filters = filters,
       token = token))

> sessionInfo()
R version 3.5.2 (2018-12-20)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Debian GNU/Linux 9 (stretch)

Matrix products: default
BLAS: /usr/lib/openblas-base/libblas.so.3
LAPACK: /usr/lib/libopenblasp-r0.2.19.so

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=C
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C
 [9] LC_ADDRESS=C               LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.0        magrittr_1.5      tidyselect_0.2.5  munsell_0.5.0
 [5] colorspace_1.4-0  R6_2.4.0          rlang_0.3.1       fansi_0.4.0
 [9] stringr_1.4.0     httr_1.4.0        plyr_1.8.4        dplyr_0.8.0.1
[13] slackr_1.4.2      tools_3.5.2       grid_3.5.2        data.table_1.12.0
[17] gtable_0.2.0      utf8_1.1.4        cli_1.0.1         askpass_1.1
[21] openssl_1.2.2     lazyeval_0.2.1    assertthat_0.2.0  tibble_2.0.1
[25] crayon_1.3.4      purrr_0.3.1       ggplot2_3.1.0     curl_3.3
[29] glue_1.3.1        stringi_1.4.3     RGA_0.4.2         compiler_3.5.2
[33] pillar_1.3.1      scales_1.0.0      jsonlite_1.6      lubridate_1.7.4
[37] pkgconfig_2.0.2

A second subsequent call with same parameters results in

-> Host: www.googleapis.com
-> User-Agent: libcurl/7.52.1 r-curl/3.3 httr/1.4.0
-> Accept-Encoding: gzip, deflate
-> Accept: application/json
->
<- HTTP/2 200
<- cache-control: no-cache, no-store, max-age=0, must-revalidate
<- date: Fri, 12 Apr 2019 08:06:19 GMT
<- expires: Mon, 01 Jan 1990 00:00:00 GMT
<- pragma: no-cache
<- content-type: application/json; charset=UTF-8
<- vary: Origin
<- vary: X-Origin
<- vary: Referer
<- content-encoding: gzip
<- server: ESF
<- content-length: 185
<- x-xss-protection: 0
<- x-frame-options: SAMEORIGIN
<- x-content-type-options: nosniff
<- alt-svc: quic=":443"; ma=2592000; v="46,44,43,39"
<-
-> GET /oauth2/v1/tokeninfo?access_token=<XXXXXXXX> HTTP/1.1
-> Host: www.googleapis.com
-> User-Agent: libcurl/7.52.1 r-curl/3.3 httr/1.4.0
-> Accept-Encoding: gzip, deflate
-> Accept: application/json
->
<- HTTP/2 200
<- cache-control: no-cache, no-store, max-age=0, must-revalidate
<- date: Fri, 12 Apr 2019 08:06:20 GMT
<- expires: Mon, 01 Jan 1990 00:00:00 GMT
<- pragma: no-cache
<- content-type: application/json; charset=UTF-8
<- vary: Origin
<- vary: X-Origin
<- vary: Referer
<- content-encoding: gzip
<- server: ESF
<- content-length: 185
<- x-xss-protection: 0
<- x-frame-options: SAMEORIGIN
<- x-content-type-options: nosniff
<- alt-svc: quic=":443"; ma=2592000; v="46,44,43,39"
<-
-> GET /analytics/v3/data/realtime?<XXXXXX> HTTP/1.1
-> Host: www.googleapis.com
-> User-Agent: libcurl/7.52.1 r-curl/3.3 httr/1.4.0
-> Accept-Encoding: gzip, deflate
-> Accept: application/json
-> Authorization: Bearer <XXXXXXX>
->
<- HTTP/2 200
<- expires: Fri, 12 Apr 2019 08:06:20 GMT
<- date: Fri, 12 Apr 2019 08:06:20 GMT
<- cache-control: private, max-age=0, must-revalidate, no-transform
<- etag: "<XXXXXXX>"
<- vary: Origin
<- vary: X-Origin
<- content-type: application/json; charset=UTF-8
<- content-encoding: gzip
<- x-content-type-options: nosniff
<- x-frame-options: SAMEORIGIN
<- x-xss-protection: 1; mode=block
<- content-length: 1568
<- server: GSE
<- alt-svc: quic=":443"; ma=2592000; v="46,44,43,39"
<-
# A tibble: XX x XX

Additional technical details

The token is created on my local machine with RGA::authorize. I changed the name of the created .rds containing the token, and injected the token in the openshift pod as a file. My R script seems to have no problem reading in this token from file with readRDS().

artemklevtsov commented 5 years ago

Thank you for the detailed report. Seems httr tries to check token and refresh it. Then httr tried to write refreshed token to file but can't do it due the permissions.

ginoa commented 5 years ago

Hi, thanks for your quick answer! Any guess as to why it fails on first call but succeeds on a second subsequent call? Suggestions on something I need to do to make sure that the first call succeeds?

artemklevtsov commented 5 years ago

System user by R-script running should have write access to the token file. I think second request was sent with the refreshed token.

ginoa commented 5 years ago

The system user running the R-script has write access to the token file. Here's the permissions to the file lrwxrwxrwx. Moreover: in order to issue a refreshed token, isn't it necessary that client.secret and client.id be loaded somewhere in the environment or RGA's own namespace? I do not inject client.secret and client.id anywhere in the machine that runs the R-script, so token refreshing should fail (unless they are included in the token.rds, I guess).

artemklevtsov / RGA