ee_Initialize() issue

[kfcabigao1]

• rgee version:1.1.3.9000
• R version: 4.2.1
• Operating System: win10

At submit an issue, please attached the following information of your rgee session:

Attach your Python (reticulate) configuration:

library(reticulate)
py_config()

python:         C:/Users/Kean Michael Cabigao/miniconda3/envs/rgee/python.exe
libpython:      C:/Users/Kean Michael Cabigao/miniconda3/envs/rgee/python38.dll
pythonhome:     C:/Users/Kean Michael Cabigao/miniconda3/envs/rgee
version:        3.8.13 | packaged by conda-forge | (default, Mar 25 2022, 05:59:00) [MSC v.1929 64 bit (AMD64)]
Architecture:   64bit
numpy:          C:/Users/Kean Michael Cabigao/miniconda3/envs/rgee/Lib/site-packages/numpy
numpy_version:  1.23.0
ee:             C:\Users\KEANMI~1\MINICO~1\envs\rgee\lib\site-packages\ee\__init__.p

Description

I followed the instructions for the installation of RGEE. However, I encountered an error when initializing. ee_check seemed to be fine as it stated OK on 3 components such as ee-api, numpy, and python version.

What I Did

I have installed gcloud as stated in Issue #267 and have already authenticated my account through gcloud init. 
The error occurred when I command rgee:: ee_Initialize(drive= T)

the error encountered is stated below:

"rgee 1.1.3.9000 ──────────────────────────────────────────────────────────────────────────────────────────────── earthengine-api 0.1.316 ── 
 ✔ user: not_defined
 ✔ Google Drive credentials:  FOUND
 ✔ Initializing Google Earth Engine:ERROR: (gcloud.auth.application-default.login) unrecognized arguments:
  Michael
  Cabigao/.config/earthengine/credentials-client-id.json
  To search the help text of gcloud commands, run:
  gcloud help -- SEARCH_TERMS
Fetching credentials using gcloud
gcloud failed. Please check for any errors above and install gcloud if needed.
Error in py_call_impl(callable, dots$args, dots$keywords) : 
  ee.ee_exception.EEException: Please authorize access to your Earth Engine account by running

earthengine authenticate

in your command line, and then retry."

Thank you~!

[kfcabigao1]

This is when I run gcloud info on cmd

Google Cloud SDK [392.0.0]

Platform: [Windows, x86_64] uname_result(system='Windows', node='LAPTOP-LHHH1RP3', release='10', version='10.0.19044', machine='AMD64') Locale: ('en_US', 'cp1252') Python Version: [3.9.12 (tags/v3.9.12:b28265d, Mar 23 2022, 23:52:46) [MSC v.1929 64 bit (AMD64)]] Python Location: [C:\Users\Kean Michael Cabigao\AppData\Local\Google\Cloud SDK\google-cloud-sdk\platform\bundledpython\python.exe] OpenSSL: [OpenSSL 1.1.1n 15 Mar 2022] Requests Version: [2.22.0] urllib3 Version: [1.25.9] Site Packages: [Disabled]

Installation Root: [C:\Users\Kean Michael Cabigao\AppData\Local\Google\Cloud SDK\google-cloud-sdk] Installed Components: bq: [2.0.75] core: [2022.06.24] gsutil: [5.10] System PATH: [C:\Users\Kean Michael Cabigao\AppData\Local\Google\Cloud SDK\google-cloud-sdk\bin..\bin\sdk;C:\Program Files\Microsoft MPI\Bin\;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:\WINDOWS\System32\OpenSSH\;C:\curl\bin;C:\Program Files\MATLAB\R2022a\bin;C:\Users\Kean Michael Cabigao\AppData\Local\Programs\Python\Python310\Scripts\;C:\Users\Kean Michael Cabigao\AppData\Local\Programs\Python\Python310\;C:\Users\Kean Michael Cabigao\AppData\Local\Microsoft\WindowsApps;C:\Program Files\snap\bin;C:\Users\Kean Michael Cabigao\AppData\Local\Google\Cloud SDK\google-cloud-sdk\bin] Python PATH: [C:\Users\Kean Michael Cabigao\AppData\Local\Google\Cloud SDK\google-cloud-sdk\lib\third_party;C:\Users\Kean Michael Cabigao\AppData\Local\Google\Cloud SDK\google-cloud-sdk\lib;C:\Users\Kean Michael Cabigao\AppData\Local\Google\Cloud SDK\google-cloud-sdk\platform\bundledpython\python39.zip;C:\Users\Kean Michael Cabigao\AppData\Local\Google\Cloud SDK\google-cloud-sdk\platform\bundledpython\DLLs;C:\Users\Kean Michael Cabigao\AppData\Local\Google\Cloud SDK\google-cloud-sdk\platform\bundledpython\lib;C:\Users\Kean Michael Cabigao\AppData\Local\Google\Cloud SDK\google-cloud-sdk\platform\bundledpython] Cloud SDK on PATH: [True] Kubectl on PATH: [False]

Installation Properties: [C:\Users\Kean Michael Cabigao\AppData\Local\Google\Cloud SDK\google-cloud-sdk\properties] User Config Directory: [C:\Users\Kean Michael Cabigao\AppData\Roaming\gcloud] Active Configuration Name: [default] Active Configuration Path: [C:\Users\Kean Michael Cabigao\AppData\Roaming\gcloud\configurations\config_default]

Account: [email] Project: [ee-kfcabigao]

Current Properties: [accessibility] screen_reader: [True] (property file) [core] account: [email] (property file) disable_usage_reporting: [False] (property file) project: [ee-kfcabigao] (property file)

Logs Directory: [C:\Users\Kean Michael Cabigao\AppData\Roaming\gcloud\logs] Last Log File: [C:\Users\Kean Michael Cabigao\AppData\Roaming\gcloud\logs\2022.06.29\16.57.00.212742.log]

git: [NOT AVAILABLE] ]sh: [OpenSSH_for_Windows_8.1p1, LibreSSL 3.0.2

[milesalanmoore]

Still experiencing a different error at ee_Initialize() as well.

[csaybar]

Hi @milesalanmoore and @kfcabigao1, can you try upgrading the earthengine-api v. 0.1.317?

library(rgee)
ee_install_upgrade() # after that restart R

[kfcabigao1]

Hello @csaybar, done upgrading the earthengine api. But the same issue persists.

"✔ Initializing Google Earth Engine:ERROR: (gcloud.auth.application-default.login) unrecognized arguments: Michael Cabigao/.config/earthengine/credentials-client-id.json To search the help text of gcloud commands, run: gcloud help -- SEARCH_TERMS Fetching credentials using gcloud Error in py_call_impl(callable, dots$args, dots$keywords) : Exception: gcloud failed. Please check for any errors above and install gcloud if needed."

I have already installed and configurated the gcloud cli and have also created a json file thru "gcloud auth application-default login" via google SDK shell.

[csaybar]

It seems that this error happens on the Python side, not in rgee, can you confirm this running in a python3 console, as follow:

import ee
ee.Authenticate()

[kfcabigao1]

Hello @csaybar,

tried to do that now in a python console but results do reveal same prompt.

"ERROR: (gcloud.auth.application-default.login) unrecognized arguments: Michael Cabigao/.config/earthengine/credentials-client-id.json To search the help text of gcloud commands, run: gcloud help -- SEARCH_TERMS Traceback (most recent call last): File "C:\Users\\miniconda3\envs\ee\Scripts\earthengine-script.py", line 9, in sys.exit(main()) File "C:\Users\\miniconda3\envs\ee\lib\site-packages\ee\cli\eecli.py", line 82, in main _run_command() File "C:\Users\\miniconda3\envs\ee\lib\site-packages\ee\cli\eecli.py", line 61, in _run_command dispatcher.run(args, config) File "C:\Users\\miniconda3\envs\ee\lib\site-packages\ee\cli\commands.py", line 356, in run self.command_dict[vars(args)[self.dest]].run(args, config) File "C:\Users\*\miniconda3\envs\ee\lib\site-packages\ee\cli\commands.py", line 398, in run ee.Authenticate(*args_auth) File "C:\Users\\miniconda3\envs\ee\lib\site-packages\ee__init__.py", line 104, in Authenticate return oauth.authenticate(authorization_code, quiet, code_verifier, auth_mode, File "C:\Users\\miniconda3\envs\ee\lib\site-packages\ee\oauth.py", line 381, in authenticate _load_app_default_credentials(auth_mode == 'gcloud', scopes, quiet) File "C:\Users\\miniconda3\envs\ee\lib\site-packages\ee\oauth.py", line 291, in _load_app_default_credentials raise Exception('gcloud failed. Please check for any errors above ' Exception: gcloud failed. Please check for any errors above and install gcloud if needed."

Do you have a step-by-step installation and configuration of Gcloud? I have configured mine based on the website but have issues Initializing.

[milesalanmoore]

@csaybar I updated to the version mentioned above via ee_install_upgrade(version = "0.1.317"). The Authorization Error window I described in #273 still appears. Python does run as instructed to test in your comment above.

[csaybar]

Hi, @milesalanmoore did you upgrade rgee too? The Authorization Error should not appear if you upgrade the earthengine-api (Python side) that is used by rgee. Can you put here the next info:

library(rgee)
reticulate::import("ee")$`__version__`

sessionInfo()

[csaybar]

@kfcabigao1 Ahh I think this should not be here Cabigao/.config/earthengine/credentials-client-id.json. You can install gcloud following these instructions: https://cloud.google.com/sdk/docs/install#deb. But before you have to delete all your credentials by running:

library(rgee)

ee_users()
ee_clean_credentials("PUT HERE YOUR USER NAME")

[milesalanmoore]

Hello @csaybar, thanks for your assistance.

reticulate::import("ee")$version [1] "0.1.317"

Here is the output of sessionInfo()

R version 4.1.2 (2021-11-01)
Platform: aarch64-apple-darwin20 (64-bit)
Running under: macOS Monterey 12.4

Matrix products: default
LAPACK: /Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods  
[7] base     

other attached packages:
[1] rgee_1.1.3

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.9        lattice_0.20-45   png_0.1-7        
 [4] ps_1.7.1          digest_0.6.29     crayon_1.5.1     
 [7] grid_4.1.2        R6_2.5.1          jsonlite_1.8.0   
[10] magrittr_2.0.3    rlang_1.0.3       cli_3.3.0        
[13] rstudioapi_0.13   leaflet_2.1.1     Matrix_1.3-4     
[16] reticulate_1.25   tools_4.1.2       htmlwidgets_1.5.4
[19] crosstalk_1.2.0   fastmap_1.1.0     compiler_4.1.2   
[22] processx_3.7.0    htmltools_0.5.2  

After updating rgee as well, a new error is introduced.

>ee_Initialize()

── rgee 1.1.4 ──────────────────────── earthengine-api 0.1.317 ── 
 ✔ user: not_defined
 ✔ Initializing Google Earth Engine:sh: gcloud: command not found
Fetching credentials using gcloud
Error in py_call_impl(callable, dots$args, dots$keywords) : 
  Exception: gcloud failed. Please check for any errors above and install gcloud if needed.

gcloud-cli is active on my terminal and authorized to access the gcloud platform via my credentials so it is strange that the glcoud command is failing now.

[kfcabigao1]

Hello @csaybar Ive tried to have a fresh install of rgee. and followed all the necessary steps. So far, I've downloaded rgee 1.1.4 with earthengine API 0.1.317. I also installed gcloud and followed the instructions on the link you've given. I also deleted all the credentials by running ee_users() and ee_clean_credentials().

the problem still persists when running ee_Initialize() as presented by the image below.

Are there steps that Ive mistakenly made? Huge thanks for being responsive in this thread.

[csaybar]

Are you available for a quite short meeting (if yes, please write me an email at csaybar@gmail.com)?. I want to take a deep look at why this sometimes happens :pray:

[h-a-graham]

Hey,

So I'm experiencing a similar issue with a colleague's Mac (x86_64). The error is the same as @milesalanmoore's https://github.com/r-spatial/rgee/issues/269#issuecomment-1185673904.

The issue appears to be related to the PATH argument for gcloud. I believe this because when running the following in the terminal: gcloud --version Everything behaves as expected.

However, system("gcloud --version"), when run in R, throws an error with "gcloud: not found".

Further, running ee.Authenticate() and ee.Initialize() directly in python work perfectly. This is within the python env created with rgee.

We have follwed the above suggestions re updating the API and latest rgee versions.

Many thanks for this package it's really appreciated! Please let me know if we can provide any further information to help solve this issue!

Cheers!

[h-a-graham]

Okay so quick update. Problems seemed to stem from having the correct permissions to write log/cache files. In our case on MacOS, this was for the "./.config/gcloud" and "./.config/earthengine" directories.

We installed gcloud from Homebrew https://formulae.brew.sh/cask/google-cloud-sdk as the PATH setting seemed to be inconsistent using Google's instructions.

Then corrected permissions for these folders, as suggested in https://github.com/Homebrew/homebrew-cask/issues/31231), by running:

sudo chown -R $(whoami):staff ~/.config/gcloud/
sudo chown -R $(whoami):staff ~/.config/earthengine/

ee_Initialize() now works as expected.

Cheers

[e-duperron]

Hi everyone,

Also had the same issue as @kfcabigao1 for this comment, without the unrecognized argument part (sorry unable to reproduce the error now).

Followed this thread successive recommandations (ee_install_upgrade(), install gcloud, check gcloud in rgee python env). I also had to install the googledrive R package (requested when running ee_Initialize() for the first time).

As suggested above by @h-a-graham gcloud indeed seemed to fail because of permission issues. For windows users, running RStudio in Administrator mode before running ee_Initialize() did the trick for me ! However the Google Drive credentials are still not working although the web page opened by gcloud did include the authorisation access to it. Still need to work this out ...

This is a quick and dirty tip to make things running, probably won't work for some people but figured it should still help some others :).

Cheers and thank you @csaybar for this awesome package !

PS : If it helps, here is my install script with all the comments and little adds additional from the installation doc.

# Installation
install.packages("rgee")
install.packages("googledrive") #required if you want to be able to use Google Drive with rgee. Used for instance to download images on your local computer

# Python environment setup (requires Python >= v3.5 ; will install Miniconda)
rgee::ee_install(py_env = "rgee") # It is just necessary once! Need to restart after and re-run in case ; path = C:/Users/`YOUR USER NAME`/AppData/Local/r-miniconda

# Checking installation non-R dependencies
rgee::ee_check()

# Upgrade the GEE API
rgee::ee_install_upgrade() #Need to restart after

# Install gcloud : follow the instructions of the web page link below. Pick all the default install options
# https://cloud.google.com/sdk/docs/install
# Can check that gcloud works within the rgee python environment : 
#     Run python.exe in C:/Users/`YOUR USER NAME`/AppData/Local/r-miniconda\envs\rgee
#     In the python console :
#         import ee
#         ee.Authenticate()
# 
# RESTART THE R SESSION AFTER INSTALLING gcloud BEFORE RUNNING THE NEXT LINES

# Initialize Earth Engine and GD :  /!\/!\/!\ NEED TO RUN RSTUDIO IN ADMINISTRATOR MODE FOR THIS LINE /!\/!\/!\
rgee::ee_Initialize(user = 'YOUR GOOGLE ACCOUNT WITH GEE', drive = TRUE, gcs = FALSE)

# Check users, credentials, path 
rgee::ee_users()
# rgee::ee_clean_credentials("YOUR GOOGLE ACCOUNT WITH GEE")

rgee::ee_get_earthengine_path()
rgee::ee_check()
rgee::ee_check_python()
rgee::ee_check_credentials()
rgee::ee_check_python_packages()

# Test code
srtm <- rgee::ee$Image("USGS/SRTMGL1_003")

viz <- list(
  max = 4000,
  min = 0,
  palette = c("#000000","#5AAD5A","#A9AD84","#FFFFFF")
)

rgee::Map$addLayer(
  eeObject = srtm,
  visParams =  viz,
  name = 'SRTM',
  # legend = TRUE
)

[GReimche]

I had the same problem here (Linux Debian11), and the answer was an incorrect gcloud installation process. Just follow the link provided bellow, and install one more time gcloud. Thank you #csaybar.

@kfcabigao1 Ahh I think this should not be here Cabigao/.config/earthengine/credentials-client-id.json. You can install gcloud following these instructions: https://cloud.google.com/sdk/docs/install#deb. But before you have to delete all your credentials by running:

library(rgee)

ee_users()
ee_clean_credentials("PUT HERE YOUR USER NAME")

[kfcabigao1]

Hi everyone,

Also had the same issue as @kfcabigao1 for this comment, without the unrecognized argument part (sorry unable to reproduce the error now).

Followed this thread successive recommandations (ee_install_upgrade(), install gcloud, check gcloud in rgee python env). I also had to install the googledrive R package (requested when running ee_Initialize() for the first time).

As suggested above by @h-a-graham gcloud indeed seemed to fail because of permission issues. For windows users, running RStudio in Administrator mode before running ee_Initialize() did the trick for me ! However the Google Drive credentials are still not working although the web page opened by gcloud did include the authorisation access to it. Still need to work this out ...

This is a quick and dirty tip to make things running, probably won't work for some people but figured it should still help some others :).

Cheers and thank you @csaybar for this awesome package !

PS : If it helps, here is my install script with all the comments and little adds additional from the installation doc.

# Installation
install.packages("rgee")
install.packages("googledrive") #required if you want to be able to use Google Drive with rgee. Used for instance to download images on your local computer

# Python environment setup (requires Python >= v3.5 ; will install Miniconda)
rgee::ee_install(py_env = "rgee") # It is just necessary once! Need to restart after and re-run in case ; path = C:/Users/`YOUR USER NAME`/AppData/Local/r-miniconda

# Checking installation non-R dependencies
rgee::ee_check()

# Upgrade the GEE API
rgee::ee_install_upgrade() #Need to restart after

# Install gcloud : follow the instructions of the web page link below. Pick all the default install options
# https://cloud.google.com/sdk/docs/install
# Can check that gcloud works within the rgee python environment : 
#     Run python.exe in C:/Users/`YOUR USER NAME`/AppData/Local/r-miniconda\envs\rgee
#     In the python console :
#         import ee
#         ee.Authenticate()
# 
# RESTART THE R SESSION AFTER INSTALLING gcloud BEFORE RUNNING THE NEXT LINES

# Initialize Earth Engine and GD :  /!\/!\/!\ NEED TO RUN RSTUDIO IN ADMINISTRATOR MODE FOR THIS LINE /!\/!\/!\
rgee::ee_Initialize(user = 'YOUR GOOGLE ACCOUNT WITH GEE', drive = TRUE, gcs = FALSE)

# Check users, credentials, path 
rgee::ee_users()
# rgee::ee_clean_credentials("YOUR GOOGLE ACCOUNT WITH GEE")

rgee::ee_get_earthengine_path()
rgee::ee_check()
rgee::ee_check_python()
rgee::ee_check_credentials()
rgee::ee_check_python_packages()

# Test code
srtm <- rgee::ee$Image("USGS/SRTMGL1_003")

viz <- list(
  max = 4000,
  min = 0,
  palette = c("#000000","#5AAD5A","#A9AD84","#FFFFFF")
)

rgee::Map$addLayer(
  eeObject = srtm,
  visParams =  viz,
  name = 'SRTM',
  # legend = TRUE
)

Hello, @e-duperron will try this method. Thank you!

[kfcabigao1]

I had the same problem here (Linux Debian11), and the answer was an incorrect gcloud installation process. Just follow the link provided bellow, and install one more time gcloud. Thank you #csaybar.

@kfcabigao1 Ahh I think this should not be here Cabigao/.config/earthengine/credentials-client-id.json. You can install gcloud following these instructions: https://cloud.google.com/sdk/docs/install#deb. But before you have to delete all your credentials by running:

library(rgee)

ee_users()
ee_clean_credentials("PUT HERE YOUR USER NAME")

Hello! @GReimche I've been following the default installation method as suggested by many window users here. But would try to re-install it once again. Thank you!

[ADLCero]

Okay so quick update. Problems seemed to stem from having the correct permissions to write log/cache files. In our case on MacOS, this was for the "./.config/gcloud" and "./.config/earthengine" directories.

We installed gcloud from Homebrew https://formulae.brew.sh/cask/google-cloud-sdk as the PATH setting seemed to be inconsistent using Google's instructions.

Then corrected permissions for these folders, as suggested in Homebrew/homebrew-cask#31231), by running:

sudo chown -R $(whoami):staff ~/.config/gcloud/
sudo chown -R $(whoami):staff ~/.config/earthengine/

ee_Initialize() now works as expected.

Cheers

Just want to say thank you to this. This worked for me. I installed via Homebrew too and followed the instructions for adding the gcloud components to the PATH as stated on their page.

[csaybar]

I think this is solved in the lastest version. Feel free to open a new issue if you consider it.