search

r-spatial / rgee

Google Earth Engine for R
https://r-spatial.github.io/rgee/
Other
666 stars 146 forks source link

ee_Initialize immediately saying credentials expired #355

Open coltonp2022 opened 7 months ago

coltonp2022 commented 7 months ago

At submit an issue, please attached the following information of your rgee session:

library(rgee)

# Initialize the Earth Engine module.
ee_Initialize()

ee_Initialize()
── rgee 1.1.7 ───────────────────────────────────────────── earthengine-api 0.1.384 ── 
 ✔ user: coltonpadilla18@gmail.com 
 ✔ Initializing Google Earth Engine:To authorize access needed by Earth Engine, open the following URL in a web browser and follow the instructions. If the web browser does not start automatically, please manually browse the URL below.

    https://code.earthengine.google.com/client-auth?scopes=https%3A//www.googleapis.com/auth/earthengine%20https%3A//www.googleapis.com/auth/devstorage.full_control&request_id=d9-FdixOIZE8Y3k-zPM1GzksROxL6XCc-9JxP1tQ83U&tc=ItjuDHONdNt0zjNudXLMPBN0pD0ngMCSWUND1uA3Zac&cc=F99GUvZM-uRj32GlRO7T7rtQQ5_dlO2MRm4FkyrIhg4

The authorization workflow will generate a code, which you should paste in the box below.
Enter verification code: 4/1AfJohXlyzYQXLojdF-CjVbyGpnVeL1dzTPEupjUSIWNS-wbU5yHCZS_yLwc

Successfully saved authorization token.
 ✔ Initializing Google Earth Engine:  DONE!
Error in value[[3L]](cond) : 
  It looks like your EE credential has expired. Try running ee_Authenticate() again or clean your credentials ee_clean_user_credentials().

Attach your Python (reticulate) configuration:

library(reticulate)
py_config()

python:         C:/Users/colto/anaconda3/envs/rgee/python.exe
libpython:      C:/Users/colto/anaconda3/envs/rgee/python38.dll
pythonhome:     C:/Users/colto/anaconda3/envs/rgee
version:        3.8.18 | packaged by conda-forge | (default, Dec 23 2023, 17:17:17) [MSC v.1929 64 bit (AMD64)]
Architecture:   64bit
numpy:          C:/Users/colto/anaconda3/envs/rgee/Lib/site-packages/numpy
numpy_version:  1.24.4
ee:             C:\Users\colto\ANACON~1\envs\rgee\lib\site-packages\ee\__init__.p

NOTE: Python version was forced by RETICULATE_PYTHON

Description

Hi, I am trying to just get initialized into rgee and for some reason it is saying that my credentials are expired immediately after I run ee_Authenticate(). I can go and find the credentials in my machine and view them, and I can find them through R as well. It is giving me an error message:

Error in value[3L] : It looks like your EE credential has expired. Try running ee_Authenticate() again or clean your credentials ee_clean_user_credentials().

I have ran both ee_Authenticate() and ee_clean_user_credentials() multiple times and have been unable to solve the issue. Can someone recreate this or point me in the right direction to solve it? The only issue that I can see happening is my computer is hooked up with Onedrive so I can work on both my personal and work computers from anywhere. I did have to move the credentials to match up with the path.expand() function; however, even that did not seem to solve the issue.

What I Did

── rgee 1.1.7 ───────────────────────────────────────────── earthengine-api 0.1.384 ── 
 ✔ user: coltonpadilla18@gmail.com 
 ✔ Initializing Google Earth Engine:To authorize access needed by Earth Engine, open the following URL in a web browser and follow the instructions. If the web browser does not start automatically, please manually browse the URL below.

    https://code.earthengine.google.com/client-auth?scopes=https%3A//www.googleapis.com/auth/earthengine%20https%3A//www.googleapis.com/auth/devstorage.full_control&request_id=d9-FdixOIZE8Y3k-zPM1GzksROxL6XCc-9JxP1tQ83U&tc=ItjuDHONdNt0zjNudXLMPBN0pD0ngMCSWUND1uA3Zac&cc=F99GUvZM-uRj32GlRO7T7rtQQ5_dlO2MRm4FkyrIhg4

The authorization workflow will generate a code, which you should paste in the box below.
Enter verification code: 4/1AfJohXlyzYQXLojdF-CjVbyGpnVeL1dzTPEupjUSIWNS-wbU5yHCZS_yLwc

Successfully saved authorization token.
 ✔ Initializing Google Earth Engine:  DONE!
Error in value[[3L]](cond) : 
  It looks like your EE credential has expired. Try running ee_Authenticate() again or clean your credentials ee_clean_user_credentials().
nozimmjon commented 7 months ago

I also bumped into the same problem...

TianyaImpression commented 7 months ago

I also bumped into the same problem... image

NicholasBerryman commented 7 months ago

I've just run into the same issue, though notably the same credentials file appears to work fine when going directly through the Python API directly in Reticulate or Jupyter. From that I've found a quick-and-dirty solution by just making a new python file with the following contents:

import ee

ee.Authenticate(auth_mode = 'localhost')
ee.Initialize()

Then replacing this R code:

...
rgee::ee_Initialize()
...

With 

...
reticulate::py_run_file('[path to python script]')
...
llechler commented 7 months ago

I have the same problem.. I authenticate gee and get the message that the authentication has been successful, but as soon as I then call ee_initialize(), I get the message that I should try running ee_authenticate and so the circle begins. Has anyone found a solution to this problem yet and could help me out?

NicholasBerryman commented 7 months ago

@llechler I've managed to get it working with the workaround I described just above your comment. Does that work for you? I'm happy to clarify if anything is unclear.

llechler commented 7 months ago

Hi @NicholasBerryman, thanks for reaching out.

I just tried it by writing the python file and then calling it in R, but I get the error message in R "Error in py_run_file_impl (file, local, convert) : ModuleNotFoundError. No module named "ee".

Some more details:

py_config() python: C:/Users/Lia/AppData/Local/r-miniconda/envs/r-reticulate/python.exe libpython: C:/Users/Lia/AppData/Local/r-miniconda/envs/r-reticulate/python38.dll pythonhome: C:/Users/Lia/AppData/Local/r-miniconda/envs/r-reticulate version: 3.8.13 | packaged by conda-forge | (default, Mar 25 2022, 05:59:00) [MSC v.1929 64 bit (AMD64)] Architecture: 64bit numpy: C:/Users/Lia/AppData/Local/r-miniconda/envs/r-reticulate/Lib/site-packages/numpy numpy_version: 1.23.4

NicholasBerryman commented 7 months ago

Hmm. Did you set your reticulate environment to the same one you use for rgee? If that workaround still doesn't work, we might have to wait for a more official solution.

llechler commented 7 months ago

I successfully managed to use both your solution and also this code snippet to initialize the GEE

reticulate::py_run_string("import ee; ee.Initialize()") reticulate::py_run_string("print(ee.version)") 0.1.385

However, I still get the error message: Error in value[3L] : rgee not initialized! Please intialize rgee. See: https://r-spatial.github.io/rgee/index.html as soon as I try to run a rgee to export a time series.. So unfortunately your workaround solution did not work for me. Unless you have an idea why, I guess I will have to wait for an "official" solution.

cofler commented 7 months ago

The workaround doesn't fully work because it doesn't create the rgee_sessioninfo.txt file (indeed the initalization goes only through Python and rgee is created from R functions)

cofler commented 7 months ago

The workaround doesn't fully work because it doesn't create the rgee_sessioninfo.txt file (indeed the initalization goes only through Python and rgee is created from R functions)

andrew-caudillo commented 7 months ago

I had this issue for quite some time. It took a lot of back and forth with google, R, and conda.

I had rgee install miniconda, even though I had conda installed. Then, I set the conda environment to that location and I have had 0 problems since. https://github.com/r-spatial/rgee/issues/353#issuecomment-1863149843

I hope this solution will help you.

jdbcode commented 7 months ago

I think there are multiple layers of things going on here.

ee_Initialize is quite complicated - lots of checks that users are setup correctly. The error message is coming from a check on a call to get user/project assets - this appears to check that a person has a home folder for assets, and if not it tries to create one for them - this is quite outdated now that Cloud projects are the standard method for asset management.

In version 0.1.383 of the earthengine-api there were changes that make specifying cloud projects more necessary. It appears to me that in ee_Initialize the ee_check_root_folder call is not successful because a specific project is not being passed during initialization, so access is blocked and we get the [probably] wrongly attributed error message about expired credentials.

I'm not sure what the fix is for the ee_Initialize function, but people can do their own authentication and initialization just using the Earth Engine client library to bypass checks that are outdated in ee_Initialize, which is what you'd do in a Jupyter notebook, for instance.

Do this instead of ee_Initialize until it is updated:

# Import rgee.
library(rgee)

# Attempt to authenticate. If credentials are found, nothing will happen except
# a return of TRUE. If credentials are not found, it'll take you through an auth
# flow and save the credentials. If you want to force reauthentication, include
# `force=TRUE` in the call. This is generally a one-time setup step.
ee$Authenticate(auth_mode='notebook')

# Initialize - this will connect to a project. You should always call this
# before working with rgee. It is IMPORTANT THAT YOU SPECIFY A PROJECT using
# the project parameter. If you forget what project IDs you have access to, find them
# here: console.cloud.google.com/project
ee$Initialize(project='YOUR-PROJECT')  # <-- EDIT THIS FOR YOUR PROJECT

# Optionally make a request to verify you are connected.
ee$String('Hello from the Earth Engine servers!')$getInfo()
btomairekirkland commented 7 months ago

The workaround allows ee_Initialise() to work for me but when running rgee functions I get the error message about the rgee_sessioninfo.txt file not being found.

csaybar commented 7 months ago

Hi all deeply sorry for all the problems regarding rgee.

I'm afraid I'm unable to continue maintaining rgee as much as I'd like to. Besides, GEE has not been very friendly to third-party development, as the legacy code isn't usually maintained. Many strong updates are made on the server side (e.g. changes in OAuth policies, etc.), We can't see these changes and we are not always informed, which makes it quite hard to keep up with the new features.

ee_Initialize was created with the idea of syncing between Drive API v3, GCS, and EE. That is the reason we did so many checks internally, unfortunately due to many changes that GEE API made over time, a lot of patches were generated.

My recommendation, like @jdbcode, is not to use ee_Initialize or even rgee.

rgee was quite useful for many years, since the only way to retrieve data was through GCS and Drive, and the idea was to make downloading easier for R users, taking advantage of the fact that the R ecosystem is much more stable than the Python one.

However, now you can move data from GEE without GGS and Drive. Actually, you have six different ways to retrieve data! (totally insane)

(I'm probably forgetting something else)

Some go faster than others, so it's best to see which one best suits your activities.

Starting this weekend, I plan to patch rgee to get a more stable version, in order to not break ongoing projects. I will begin deprecating ee_install, ee_Initialize and download&upload functions (e.g. ee_as_raster) because again there are now more efficient solutions.

It is important to note that the GEE can change unexpectedly, and I can bet they will. Therefore, if you require a more dependable option for your applications or projects in the long run, the best choice by far would be to use rstac and a STAC data server.

El lun, 22 ene 2024 a las 11:32, Máire Kirkland @.***>) escribió:

The workaround allows ee_Initialise() to work for me but when running rgee functions I get the error message about the rgee_sessioninfo.txt file not being found.

— Reply to this email directly, view it on GitHub https://github.com/r-spatial/rgee/issues/355#issuecomment-1903812398, or unsubscribe https://github.com/notifications/unsubscribe-auth/AD752PXRZJ3DEVOISEXRWF3YPZE3PAVCNFSM6AAAAABBNR2FJKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMBTHAYTEMZZHA . You are receiving this because you are subscribed to this thread.Message ID: @.***>

jordi-1 commented 7 months ago

Dear Cesar,

I get that you might feel discouraged. I would like to add that while just being a normal R user for epidemiology, I have always managed, even recently, to find ways to use rgee to access GEE data. Of course, there were times when fixing the install was longer than others but relying on an interface in a language (R) that I can understand is a major advantage. You are mentionning multiple ways to access earthengine data, however, for a standard epidemiologist, none are as clear as rgee. I don't think I am the only one in this situation... Thanks again for all the help provided by your package, and please reconsider and keep it alive !!! Long live rgee !

TianyaImpression commented 6 months ago

rgee is really very easy to use. R language environment + GEE is really a magic weapon for remote sensing data analysis. It is a pity to give up rgee. I hope rgee can continue to update. Thanks again for all the help provided by your package, and please reconsider and keep it alive !!! Long live rgee !

Gelsleichter commented 5 months ago

I am running in windows machine, R version 4.3.3. Python version 3.8.18 (conda with reticulate)

It worked when I downgrade the GEE API from version 0.1.391 to 0.1.370, see details in the comment: https://github.com/r-spatial/rgee/issues/353#issuecomment-1983765552

Cidree commented 5 months ago

I am running in windows machine, R version 4.3.3. Python version 3.8.18 (conda with reticulate)

It worked when I downgrade the GEE API from version 0.1.391 to 0.1.370, see details in the comment: #353 (comment)

Thank you! This worked for me

rion-saeon commented 4 months ago

I am running in windows machine, R version 4.3.3. Python version 3.8.18 (conda with reticulate)

It worked when I downgrade the GEE API from version 0.1.391 to 0.1.370, see details in the comment: #353 (comment)

This also worked for me!

MarShaikh commented 4 months ago

The workaround allows ee_Initialise() to work for me but when running rgee functions I get the error message about the rgee_sessioninfo.txt file not being found.

Sorry for referencing your comment after many months, but in case someone else gets into this problem after trying the workaround. I found a very dirty workaround to the workaround (I know, insane). Just set up your workaround as described above, and after the .py file call, write the statement rgee::ee_Initialise().

Explanation?: If I understand correctly, RGEE through python is able to do the initialization but since we are doing it through python, there is no rgee_sessioninfo.txt file generated. This file can then be generated through R's initialization command. Just a theory, but this worked for me.

judgementc commented 3 months ago

The workaround allows ee_Initialise() to work for me but when running rgee functions I get the error message about the rgee_sessioninfo.txt file not being found.

Sorry for referencing your comment after many months, but in case someone else gets into this problem after trying the workaround. I found a very dirty workaround to the workaround (I know, insane). Just set up your workaround as described above, and after the .py file call, write the statement rgee::ee_Initialise().

_Explanation?: If I understand correctly, RGEE through python is able to do the initialization but since we are doing it through python, there is no rgee_sessioninfo.txt file generated. This file can then be generated through R's initialization command. Just a theory, but this worked for me._

Hi, your suggestion is very useful, but I can't initialize the regee with python first, so I can't generate a regee_sessioninfo.txt. Can you please tell me how to manually create a rgee_sessioninfo.txt and put it in the right place?

MarShaikh commented 3 months ago

The workaround allows ee_Initialise() to work for me but when running rgee functions I get the error message about the rgee_sessioninfo.txt file not being found.

Sorry for referencing your comment after many months, but in case someone else gets into this problem after trying the workaround. I found a very dirty workaround to the workaround (I know, insane). Just set up your workaround as described above, and after the .py file call, write the statement rgee::ee_Initialise(). _Explanation?: If I understand correctly, RGEE through python is able to do the initialization but since we are doing it through python, there is no rgee_sessioninfo.txt file generated. This file can then be generated through R's initialization command. Just a theory, but this worked for me._

Hi, your suggestion is very useful, but I can't initialize the regee with python first, so I can't generate a regee_sessioninfo.txt. Can you please tell me how to manually create a rgee_sessioninfo.txt and put it in the right place?

Hi, sorry for not being clear. rgee_sessioninfo.txt is a file used by the rgee package in R. So you won't be able to generate it using Python. The only way to generate it is by using rgee::ee_Initialize() function call from your R script.

Leprechault commented 2 months ago

For me, it doesn't work:

 reticulate::py_run_file('/home/fores/anaconda3/envs/myyolov8/GEE_auth.py')
Error in py_run_file_impl(file, local, convert) : 
  ModuleNotFoundError: No module named 'StringIO'
Run `reticulate::py_last_error()` for details.

and in my GEE_auth.pyfile I have:

import ee
import io
from io import StringIO

ee.Authenticate(auth_mode = 'localhost')
ee.Initialize()

or:

import ee

ee.Authenticate(auth_mode = 'localhost')
ee.Initialize()
andrew-caudillo commented 2 months ago

For me, it doesn't work:

 reticulate::py_run_file('/home/fores/anaconda3/envs/myyolov8/GEE_auth.py')
Error in py_run_file_impl(file, local, convert) : 
  ModuleNotFoundError: No module named 'StringIO'
Run `reticulate::py_last_error()` for details.

and in my GEE_auth.pyfile I have:

import ee
import io
from io import StringIO

ee.Authenticate(auth_mode = 'localhost')
ee.Initialize()

or:

import ee

ee.Authenticate(auth_mode = 'localhost')
ee.Initialize()

Have you tried just reinstalling both rgee and reticulate? Sometimes I had to do this (a few times) for the packages to work.