rgee::ee_Initialize() actually de-authenticates me?

[MatthieuStigler]

I am authenticated to earthengine, but once I run rgee::ee_Initialize(user =xxx'), it seems to disconnect me, or de-authenticate me even on the command line?

See below, where I am authenticated (i.e. earthengine ls returns something valid), I can run rgee::ee$Initialize() and still get a valid earthengine ls, but once I run rgee::ee_Initialize(user =xxx') it fails and then even earthengine ls fails?

> packageVersion("rgee")
[1] ‘1.1.5’
> system("~/programs/anaconda3/envs/ee/bin/earthengine ls")
projects/earthengine-legacy/assets/users/xxx
> rgee::ee$Initialize()
> system("~/programs/anaconda3/envs/ee/bin/earthengine ls")
projects/earthengine-legacy/assets/users/xxx
> rgee::ee_Initialize(user = 'xxx@gmail.com')
── rgee 1.1.5 ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────── earthengine-api 0.1.248 ── 
 ✔ user: xxx@gmail.com 
 ✔ Initializing Google Earth Engine:Error in py_call_impl(callable, dots$args, dots$keywords) : 
  google.auth.exceptions.RefreshError: ('invalid_grant: Bad Request', {'error': 'invalid_grant', 'error_description': 'Bad Request'})
In addition: Warning message:
In rgee::ee_Initialize(user = "xxx@gmail.com") :

 Error in py_call_impl(callable, dots$args, dots$keywords) : 
google.auth.exceptions.RefreshError: ('invalid_grant: Bad Request', {'error': 'invalid_grant', 'error_description': 'Bad Request'}) 
> system("~/programs/anaconda3/envs/ee/bin/earthengine ls")
Traceback (most recent call last):
  File "/home/user/programs/anaconda3/envs/ee/bin/earthengine", line 10, in <module>
    sys.exit(main())
  File "/home/user/programs/anaconda3/envs/ee/lib/python3.9/site-packages/ee/cli/eecli.py", line 82, in main
    _run_command()
  File "/home/user/programs/anaconda3/envs/ee/lib/python3.9/site-packages/ee/cli/eecli.py", line 61, in _run_command
    dispatcher.run(args, config)
  File "/home/user/programs/anaconda3/envs/ee/lib/python3.9/site-packages/ee/cli/commands.py", line 355, in run
    self.command_dict[vars(args)[self.dest]].run(args, config)
  File "/home/user/programs/anaconda3/envs/ee/lib/python3.9/site-packages/ee/cli/commands.py", line 802, in run
    config.ee_init()
  File "/home/user/programs/anaconda3/envs/ee/lib/python3.9/site-packages/ee/cli/utils.py", line 131, in ee_init
    ee.Initialize(
  File "/home/user/programs/anaconda3/envs/ee/lib/python3.9/site-packages/ee/__init__.py", line 132, in Initialize
    data.initialize(
  File "/home/user/programs/anaconda3/envs/ee/lib/python3.9/site-packages/ee/data.py", line 201, in initialize
    _install_cloud_api_resource()
  File "/home/user/programs/anaconda3/envs/ee/lib/python3.9/site-packages/ee/data.py", line 260, in _install_cloud_api_resource
    _cloud_api_resource = _cloud_api_utils.build_cloud_resource(
  File "/home/user/programs/anaconda3/envs/ee/lib/python3.9/site-packages/ee/_cloud_api_utils.py", line 166, in build_cloud_resource
    resource = build()
  File "/home/user/programs/anaconda3/envs/ee/lib/python3.9/site-packages/ee/_cloud_api_utils.py", line 146, in build
    return discovery.build(
  File "/home/user/programs/anaconda3/envs/ee/lib/python3.9/site-packages/googleapiclient/_helpers.py", line 134, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "/home/user/programs/anaconda3/envs/ee/lib/python3.9/site-packages/googleapiclient/discovery.py", line 270, in build
    content = _retrieve_discovery_doc(
  File "/home/user/programs/anaconda3/envs/ee/lib/python3.9/site-packages/googleapiclient/discovery.py", line 376, in _retrieve_discovery_doc
    resp, content = req.execute(num_retries=num_retries)
  File "/home/user/programs/anaconda3/envs/ee/lib/python3.9/site-packages/googleapiclient/_helpers.py", line 134, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "/home/user/programs/anaconda3/envs/ee/lib/python3.9/site-packages/googleapiclient/http.py", line 900, in execute
    resp, content = _retry_request(
  File "/home/user/programs/anaconda3/envs/ee/lib/python3.9/site-packages/googleapiclient/http.py", line 177, in _retry_request
    resp, content = http.request(uri, method, *args, **kwargs)
  File "/home/user/programs/anaconda3/envs/ee/lib/python3.9/site-packages/google_auth_httplib2.py", line 209, in request
    self.credentials.before_request(self._request, method, uri, request_headers)
  File "/home/user/programs/anaconda3/envs/ee/lib/python3.9/site-packages/google/auth/credentials.py", line 133, in before_request
    self.refresh(request)
  File "/home/user/programs/anaconda3/envs/ee/lib/python3.9/site-packages/google/oauth2/credentials.py", line 293, in refresh
    ) = reauth.refresh_grant(
  File "/home/user/programs/anaconda3/envs/ee/lib/python3.9/site-packages/google/oauth2/reauth.py", line 338, in refresh_grant
    _client._handle_error_response(response_data)
  File "/home/user/programs/anaconda3/envs/ee/lib/python3.9/site-packages/google/oauth2/_client.py", line 60, in _handle_error_response
    raise exceptions.RefreshError(error_details, response_data)
google.auth.exceptions.RefreshError: ('invalid_grant: Bad Request', {'error': 'invalid_grant', 'error_description': 'Bad Request'})

[csaybar]

Hi @MatthieuStigler

rgee save the user credentials in subfolders inside .config/earthengine/. The problem seems that the credentials you saved in the past are no longer valid. Therefore you need to delete the credentials because rgee will overwrite the new ones.

ee_clean_credentials("your_user")
ee_Initialize()

I'm in the middle of rewriting the auth of rgee, since now it has a lot of conflict with the latest versions of earthengine-api. the user argument will be deprecated.

[MatthieuStigler]

thanks for your quick answer, will test tonight.

So if I understood you well, rgee is storing its own credentials elsewhere, then copying them to .config/earthengine/? I assume this is also the folder where earthengine authenticate stores the credentials? So this would explain why the new valid credentials are overwritten by the old one?

Thanks!

[csaybar]

Yes exactly! I'm preparing a new rgee version without this weird behaviour. You can check the new auth in r-spatial/rgee@newauth. I just checked the code below in the Rstudio cloud, and it works perfectly!

install.packages(c("remotes", "googledrive"))
remotes::install_github("r-spatial/rgee@newauth")

library(rgee)

ee_install()

# Run in a terminal ----------

# system("curl -sSL https://sdk.cloud.google.com | bash")
# PATH=$PATH:/home/rstudio-user/google-cloud-sdk/bin/
# earthengine authenticate --quiet

ee_Initialize(drive = TRUE)

[zackarno]

I am having this issue around 1x per week now. The ee_clean_credentials() work around does the trick, but is there a way to minimize having to do this?

[csaybar]

Hi, @zackarno

We decided to change the default to auth_mode="notebook" (as geemap does) to avoid all the issues opened with gcloud no found (gcloud is the default in earthengine-api). I was very much against doing this because you only need to define the gcloud path in your $PATH. However, I think it can be a bit hard for new R/GoogleCloud users, so I decided to do it simply, and use auth_mode="notebook" (see: https://developers.google.com/earth-engine/apidocs/ee-authenticate)

The trade-off is that your credential will expire if I understand correctly (the documentation is not very clear here, I come to that conclusion based on my experience with GEE). I can think of two workarounds.

• Option 1:

  library(rgee)
  ee_Authenticate(earthengine = TRUE, drive = FALSE,   gcs = FALSE)  # ask again for earthengine credentials

• Option 2:

library(rgee)

# install and set gcloud on your computer and then
ee_Authenticate(earthengine = TRUE, auth_mode="gcloud")  # None == 'gcloud'  in local environments

# or simply
ee_Initialize(auth_mode="gcloud")

In theory, the credentials with gcloud shouldn't expire, but sometimes it does too (not sure but one day my credentials stop working after 5 moths, maybe I did something). Another option is to remove your credentials using ee_clean_user_credentials.

I hope this has sense to you, let me know if you have an idea how to solve this issue with authentication. All these changes appeared after google applied more security to the initialization.

[zackarno]

Hi @csaybar - thanks for the quick response and sorry for the delay getting back to you. That makes sense. I think I had been doing the initializing without specifying the auth_mode and therefore not been using the gcloud authentication. I had set up the gcloud and all seemed to be working fine so I was not aware of the auth_mode="notebook" option even though I had been doing it by default. I just switched back to auth_mode="gcloud" following your instructions under option 2 (thanks!!) so hopefully that should keep the credentials from expiring as frequently.

I've set up rgee on many computers and generally don't face major issues, but I have seen colleagues struggle and I believe gcloud could be one of the sticking points. These issues also tend to be relatively difficult to help trouble shoot remotely, so better understanding this notebook vs gcloud option seems really useful -- thank you very much!

[dhklmilan]

I’m having a bit of a frustrating issue with the Google Earth Engine API. After initializing it successfully, I get this warning message: ✔ Initializing Google Earth Engine: DONE! credentials are cached in the path: C:\Users\dklmn/.config/earthengine/ Warning messages: 1: In ee_check_init() : Update your earthengine-api installations to v0.1.317 or greater. Earlier versions are not compatible with recent changes to the Earth Engine backend. 2: In ee_check_init() : Update your earthengine-api installations to v0.1.317 or greater. Earlier versions are not compatible with recent changes to the Earth Engine backend. Successfully saved authorization token. I already have earthengine-api version 1.1.1 installed.

ee_Initialize(auth_mode="gcloud") ── rgee 1.1.7 ──────────────────────────────────────────── earthengine-api 1.1.1 ── ✔ user: not_defined ✔ Initializing Google Earth Engine: DONE! Error in value[3L] : It looks like your EE credential has expired. Try running ee_Authenticate() again or clean your credentials ee_clean_user_credentials(). In addition: Warning messages: 1: In ee_check_init() : Update your earthengine-api installations to v0.1.317 or greater. Earlier versions are not compatible with recent changes to the Earth Engine backend. 2: In ee_check_init() : Update your earthengine-api installations to v0.1.317 or greater. Earlier versions are not compatible with recent changes to the Earth Engine backend.