rgee does not work with drive (edit: sometimes?!)

jakemanger commented 5 months ago

rgee version:
R version:
Operating System:

At submit an issue, please attached the following information of your `rgee` session:

[x] You have the Python API installed (from terminal):
```
earthengine -h
```

[x] You can find the credentials file on your system:

library(rgee)
ee_path <- path.expand("~/.config/earthengine/credentials")
file.exists(ee_path)

[x] You can run a simple EE command from R:

library(rgee)

# Initialize the Earth Engine module.
ee_Initialize()

# Print metadata for a DEM dataset.
print(ee$Image('USGS/SRTMGL1_003')$getInfo())

Attach your Python (reticulate) configuration:

> library(reticulate)
Warning message:
package ‘reticulate’ was built under R version 4.2.3 
> py_config()
python:         C:/Users/00057/AppData/Local/r-miniconda/envs/rgee/python.exe
libpython:      C:/Users/00057/AppData/Local/r-miniconda/envs/rgee/python38.dll
pythonhome:     C:/Users/00057/AppData/Local/r-miniconda/envs/rgee
version:        3.8.17 | packaged by conda-forge | (default, Jun 16 2023, 07:01:59) [MSC v.1929 64 bit (AMD64)]
Architecture:   64bit
numpy:          C:/Users/00057/AppData/Local/r-miniconda/envs/rgee/Lib/site-packages/numpy
numpy_version:  1.24.4
ee:             C:\Users\00057\AppData\Local\R-MINI~1\envs\rgee\lib\site-packages\ee\__init__.p

NOTE: Python version was forced by RETICULATE_PYTHON

Description

EDIT: This problem seems to have disappeared on it's own rerunning the code a week later. Could this be a server-side problem, e.g. rate limiting or GEE being too heavily used at the time?

If I try and extract data with the google drive option, the extraction lags forever. However, if I try the getInfo option, it is almost instantaneous. This happens with every dataset I've tried. I've attached the example from the docs found at https://r-spatial.github.io/rgee/reference/ee_extract.html, below.

What I Did

The example found at https://r-spatial.github.io/rgee/reference/ee_extract.html

library(rgee)
library(sf)

ee_Initialize(gcs = TRUE, drive = TRUE)

# Define a Image or ImageCollection: Terraclimate
terraclimate <- ee$ImageCollection("IDAHO_EPSCOR/TERRACLIMATE") %>%
 ee$ImageCollection$filterDate("2001-01-01", "2002-01-01") %>%
ee$ImageCollection$map(
   function(x) {
     date <- ee$Date(x$get("system:time_start"))$format('YYYY_MM_dd')
     name <- ee$String$cat("Terraclimate_pp_", date)
     x$select("pr")$rename(name)
   }
 )

# Define a geometry
nc <- st_read(
 dsn = system.file("shape/nc.shp", package = "sf"),
 stringsAsFactors = FALSE,
 quiet = TRUE
)

#Extract values - getInfo
# this works fine and very fast
ee_nc_rain <- ee_extract(
 x = terraclimate,
 y = nc["NAME"],
 scale = 250,
 fun = ee$Reducer$mean(),
 sf = TRUE
)

# Extract values - drive (lazy = TRUE)
# this starts and never finishes
ee_nc_rain <- ee_extract(
 x = terraclimate,
 y = nc["NAME"],
 scale = 250,
 fun = ee$Reducer$mean(),
 via = "drive",
 lazy = TRUE,
 sf = TRUE
)
ee_nc_rain <- ee_nc_rain %>% ee_utils_future_value()

jakemanger commented 5 months ago

Apologies, but it seems like perhaps GEE was down at the time of originally running this, or I was rate limited or something similar. Is there a way to check if that is happening? I appreciate your time

jdbcode commented 4 months ago

If I try and extract data with the google drive option, the extraction lags forever. However, if I try the getInfo option, it is almost instantaneous.

Earth Engine has two processing environments: interactive and batch. Interactive is getInfo, it is meant for small requests that can be returned within 5 minutes. Batch is for exporting to Drive, Cloud Storage, Asset. It takes longer, but can scale to arbitrarily large tasks.

Learn more here: https://developers.google.com/earth-engine/guides/processing_environments

r-spatial / rgee