ee_as_sf(via=gcs) leads to Cannot update access control for an object when uniform bucket-level access is enabled

[MatthieuStigler]

• rgee version: 1.1.2
• R version: 4.1.2
• Operating System: Ubuntu 20.04

When I try to replicate the example in ee_as_sf(), using via="gcs", I get a strange error message: _Error in abort_http(): ! http400 Cannot update access control for an object when uniform bucket-level access is enabled. Strangely enough, the task goes through and shows as completed. Should I have used a different type of control access for that bucket on google cloud storage?

Thanks!

Code:

library(rgee)

ee_Initialize(drive = TRUE, gcs = TRUE)
#> ── rgee 1.1.2 ─────────────────────────────────────── earthengine-api 0.1.248 ── 
#>  ✓ Google Drive credentials: ✓ Google Drive credentials:  FOUND
#>  ✓ GCS credentials: ✓ GCS credentials:  FOUND
#>  ✓ Initializing Google Earth Engine: ✓ Initializing Google Earth Engine:  DONE!
#> ────────────────────────────────────────────────────────────────────────────────

# Region of interest
roi <- ee$Geometry$Polygon(list(
  c(-122.275, 37.891),
  c(-122.275, 37.868),
  c(-122.240, 37.868),
  c(-122.240, 37.891)
))

# TIGER: US Census Blocks Dataset
blocks <- ee$FeatureCollection("TIGER/2010/Blocks")
subset <- blocks$filterBounds(roi)

# Download via GCS
sf_randomPoints_gcs <- ee_as_sf(
  x = subset,
  via = 'gcs',
  container = 'rgee-tests' #GCS bucket name
)
#> Registered S3 method overwritten by 'geojsonsf':
#>   method        from   
#>   print.geojson geojson
#> 
#> - download parameters (Google Drive)
#>  Table ID    : Blocks 
#>  Folder name : rgee-tests 
#>  Date        : 2022_02_15_20_42_35 
#> Polling for task <id: A7YM6BAJILZZBEXEFDJY7EZH, time: 0s>.
#> Polling for task <id: A7YM6BAJILZZBEXEFDJY7EZH, time: 5s>.
#> Polling for task <id: A7YM6BAJILZZBEXEFDJY7EZH, time: 10s>.
#> Polling for task <id: A7YM6BAJILZZBEXEFDJY7EZH, time: 15s>.
#> Polling for task <id: A7YM6BAJILZZBEXEFDJY7EZH, time: 20s>.
#> Polling for task <id: A7YM6BAJILZZBEXEFDJY7EZH, time: 25s>.
#> Polling for task <id: A7YM6BAJILZZBEXEFDJY7EZH, time: 30s>.
#> Polling for task <id: A7YM6BAJILZZBEXEFDJY7EZH, time: 35s>.
#> Polling for task <id: A7YM6BAJILZZBEXEFDJY7EZH, time: 40s>.
#> State: COMPLETED
#> ℹ 2022-02-15 20:43:26 > Request Status Code:  400
#> Error in `abort_http()`:
#> ! http_400 Cannot update access control for an object when uniform bucket-level access is enabled. Read more at https://cloud.google.com/storage/docs/uniform-bucket-level-access

And using rlang::last_error() shows:

rlang::last_error()
<error/http_400>
Error in `abort_http()`:
! http_400 Cannot update access control for an object when uniform bucket-level access is enabled. Read more at https://cloud.google.com/storage/docs/uniform-bucket-level-access
Backtrace:
 1. rgee::ee_as_sf(x = subset, via = "gcs", container = "rgee-tests")
 2. rgee:::ee_sf_gcs_local(...)
 3. rgee::ee_gcs_to_local(...)
 4. googleCloudStorageR::gcs_update_object_acl(...)
 5. googleAuthR insert(path_arguments = list(b = bucket, o = object_name), the_body = accessControls)
 6. googleAuthR:::doHttrRequest(...)
 7. googleAuthR:::retryRequest(...)
 8. googleAuthR:::abort_http(status_code, error)

^{Created on 2022-02-15 by the reprex package (v2.0.1)}

[csaybar]

Hi @MatthieuStigler you have to switch your rgee-tests bucket from bucket-level to fine-grained level. A fully tutorial about how to do this and why is here: https://r-spatial.github.io/rgee/articles/rgee05.html.

[csaybar]

Please check if your SaK is working perfectly, running:

# remotes::install_github("r-spatial/rgee") Install rgee v.1.3
library(rgee)

ee_Initialize("csaybar")

SaK_file <- "/home/csaybar/Downloads/SaK_rgee.json" # PUT HERE THE FULLNAME OF YOUR SAK.

# Assign the SaK to a EE user.
ee_utils_sak_copy(
  sakfile =  SaK_file,
  users = c("csaybar", "ryali93") # Unlike GD, we can use the same SaK for multiple users.
)

# Validate your SaK
ee_utils_sak_validate()

[MatthieuStigler]

oh, sorry about this, I missed that part of the documentation! I changed the access control, and now it runs fine!

I also tried the ee_utils_sak_validate() function, it does not work when I don't specify bucket (given that some buckets have the previous uniform control behavior), but it does when I specify ee_utils_sak_validate(bucket = "rgee-tests")

[MatthieuStigler]

As a side note, if one tries ee_utils_sak_validate(bucket = "wrong"), some error messages come very early, but the test still takes a minute?

ee_utils_sak_validate(bucket = "wrong")
── SaK validator ───────────────────────────────────────────────────────────────────────────── The test should take ~1 min. Please wait. ──
http_403 The project to be billed is associated with an absent billing account.
An ERROR raised when rgee tries to write in your GCS bucket.
http_403 Unspecified error
An ERROR raised when rgee tries to read your GCS bucket.
ERROR in Earth Engine servers: Unable to write to bucket wrong (permission denied).
ee_monitoring was forced to stop before getting results
An ERROR raised when rgee tries to sync GEE & GCS.
Error in `abort_http()`:
! http_404 No such object: wrong/demo_data.csv
Run `rlang::last_error()` to see where the error occurred.

[csaybar]

Hi @MatthieuStigler can you show us the message error when run ee_utils_sak_validate().

As a side note, if one tries ee_utils_sak_validate(bucket = "wrong"), some error messages come very early, but the test still takes a minute?

You're right, there is no reason for continuous checking if a previous check fails, we will fix it soon!