search

tidyverse / googledrive

Google Drive R API
https://googledrive.tidyverse.org/
Other
322 stars 47 forks source link

Authorization has recently broken #440

Closed quantitative-technologies closed 1 year ago

quantitative-technologies commented 1 year ago

I had been using non-interactive auth without issue for some time. I work in the cloud and currently interactive auth is not possible (at least I'm not sure how to do it).

It stopped working recently. Here is an MRE demonstrating an issue, given that my token is saved in the cache. I am running R from the command line:

> options(gargle_verbosity = "debug")
> options(gargle_oauth_cache = ".secrets")
> options(gargle_oauth_email = TRUE)
> library(googledrive)
initializing AuthState
> drive_auth()
trying `token_fetch()`
Trying `credentials_byo_oauth()` ...
Error caught by `token_fetch()`:
inherits(token, "Token2.0") is not TRUE
trying `credentials_service_account()`
Error caught by `token_fetch()`:
Argument 'txt' must be a JSON string, URL or file.
trying `credentials_external_account()`
aws.ec2metadata not installed; can't detect whether running on EC2 instance
trying `credentials_app_default()`
Trying `credentials_gce()` ...
Error caught by `token_fetch()`:
Expected content type application/json, not application/text.
trying `credentials_user_oauth2()`
attempt to access internal gargle data from: googledrive
Gargle2.0 initialize
adding "userinfo.email" scope
loading token from the cache
email: '*'
oauth client name: tidyverse-clio
oauth client name: installed
oauth client id: 603366585132-dpeg5tt0et3go5of2374d83ifevk5086.apps.googleusercontent.com
scopes: ...drive, ...userinfo.email
token(s) found in cache:
b2029c2d434730bf3b86215aa512b190_james.hirschorn@quantitative-technologies.com
token we are looking for:
no matching token in the cache
initiating new token
Waiting for authentication in browser...
Press Esc/Ctrl + C to abort
/usr/bin/xdg-open: 882: www-browser: not found
/usr/bin/xdg-open: 882: links2: not found
/usr/bin/xdg-open: 882: elinks: not found
/usr/bin/xdg-open: 882: links: not found
/usr/bin/xdg-open: 882: lynx: not found
/usr/bin/xdg-open: 882: w3m: not found
xdg-open: no method available for opening 'https://accounts.google.com/o/oauth2/v2/auth?client_id=603366585132-dpeg5tt0et3go5of2374d83ifevk5086.apps.googleusercontent.com&scope=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fdrive%20https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fuserinfo.email&redirect_uri=http%3A%2F%2Flocalhost%3A1410%2F&response_type=code&state=7d44daac84410b8f75c5937785a70827'

You can see that there is "no matching token in the cache", but there is only one token in the cache and according to the docs it should match since I have set gargle_oauth_email = TRUE.

> sessionInfo()
R version 4.3.1 (2023-06-16)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 22.04.2 LTS

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 
LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.20.so;  LAPACK version 3.10.0

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

time zone: Etc/UTC
tzcode source: system (glibc)

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] googledrive_2.1.1

loaded via a namespace (and not attached):
 [1] later_1.3.1      utf8_1.2.3       R6_2.5.1         httpuv_1.6.11   
 [5] tidyselect_1.2.0 magrittr_2.0.3   glue_1.6.2       tibble_3.2.1    
 [9] pkgconfig_2.0.3  generics_0.1.3   dplyr_1.1.2      lifecycle_1.0.3 
[13] promises_1.2.0.1 cli_3.6.1        askpass_1.1      fansi_1.0.4     
[17] openssl_2.1.0    gargle_1.5.2     vctrs_0.6.3      compiler_4.3.1  
[21] httr_1.4.6       purrr_1.0.1      curl_5.0.1       pillar_1.9.0    
[25] Rcpp_1.0.11      jsonlite_1.8.7   rlang_1.1.1      fs_1.6.3
jennybc commented 1 year ago

I can tell you are using an OAuth client of the "installed" or "Desktop app" type, which is likely no longer going to work in a cloud setting.

oauth client name: tidyverse-clio
oauth client name: installed

(that second debugging message should read "oauth client type", oops)

Can you say more about the context and computing environment?

What is the process you are using to put that token into the .secrets cache?

Overall this feels like you've gotten a token in one (non-cloud?) context and are then trying to use it in a cloud context. And since we default to OAuth clients of a different type in those two settings, it's causing the cache miss. I just need to understand your situation better to advise on what to do.

quantitative-technologies commented 1 year ago

Ah, I misunderstood the documentation. I interpreted it to mean that if there is a single token in the cache then it must match.

You are correct about my process. The token was generated using googledrive on my local machine, and I just added it to .secrets using git. Then it is failing to match in the cloud. I cannot generate the token in the cloud anymore, because Rstudio server no longer works there.

Is there a way to generate a token locally that will match, or to instruct the cloud to accept the locally generated token?

jennybc commented 1 year ago

In the code that needs to use a cached token (originally obtained on your local computer) in a cloud context, you'll want to set some options:

options(
  gargle_oauth_cache = ".secrets",
  gargle_oauth_email = TRUE,
  gargle_oauth_client_type = "installed"
)

Or you can intentionally use the OAuth client of "web" type locally, which is a clunkier flow, but still works, when you first capture the token:

options(
  gargle_oauth_cache = ".secrets",
  gargle_oauth_client_type = "web"
)
# obtain a token interactively HERE

in which case you would delete gargle_oauth_client_type = "installed" from the code running in the cloud (because the default "web" is what you want). Or have gargle_oauth_client_type = "web" everywhere just to remind yourself what you're doing.

quantitative-technologies commented 1 year ago

The first approach was unsuccessful, adding the option gargle_oauth_client_type = "installed" still misses the cached token:

> files <- drive_ls(instances_id)
attempt to access internal gargle data from: googledrive
trying `token_fetch()`
Trying `credentials_byo_oauth()` ...
Error caught by `token_fetch()`:
inherits(token, "Token2.0") is not TRUE
trying `credentials_service_account()`
Error caught by `token_fetch()`:
Argument 'txt' must be a JSON string, URL or file.
trying `credentials_external_account()`
aws.ec2metadata not installed; can't detect whether running on EC2 instance
trying `credentials_app_default()`
Trying `credentials_gce()` ...
Error caught by `token_fetch()`:
Expected content type application/json, not application/text.
trying `credentials_user_oauth2()`
attempt to access internal gargle data from: googledrive
Gargle2.0 initialize
adding "userinfo.email" scope
loading token from the cache
email: '*'
oauth client name: tidyverse-clio
oauth client name: installed
oauth client id: 603366585132-dpeg5tt0et3go5of2374d83ifevk5086.apps.googleusercontent.com
scopes: ...drive, ...userinfo.email
token(s) found in cache:
134f22af3ae3a9f0b7f0eb57dd61916f_james.hirschorn@quantitative-technologies.com
token we are looking for:
no matching token in the cache
initiating new token

Since rstudio server is working again (in the cloud), I generated a new token interactively in the cloud.

I also need to use this noninteractively in the cloud, so I put the option gargle_oauth_client_type = "web" in my script (not sure if this should be necessary?) and everything seems OK now.

jennybc commented 1 year ago

The bottom line is that the OAuth client needs to match in these two contexts (along with other token details, such as the target user, scopes, etc.):

The tricky bit to appreciate is that the default OAuth client varies with the computing environment:

So, to ensure client matching, you probably need to explicitly specify the client type on one end, to match the other.

If you've got something working, then godspeed.

quantitative-technologies commented 1 year ago

Well, it seems to have solved my issue. I can use my token with noninteractive authentication is two different contexts by specifying the client type.