GlobalFishingWatch / gfwr

R package for accessing data from Global Fishing Watch APIs
https://globalfishingwatch.github.io/gfwr/
Apache License 2.0
59 stars 7 forks source link

`get_raster`, geometry too large #173

Closed tom-lvn closed 3 weeks ago

tom-lvn commented 1 month ago

Hey gfwr team,

I am trying to use the get_raster() function with a custom sf shapefile but I receive the following error:

med_effort = get_raster(
  spatial_resolution = 'HIGH',
  temporal_resolution = 'DAILY',
  group_by = 'GEARTYPE',
  start_date = '2021-01-01',
  end_date = '2021-06-30',
  region = med_sf,
  region_source = 'USER_SHAPEFILE',
  key = gfw_auth()
)
`Error in `httr2::req_perform()`:
! HTTP 422 Unprocessable Entity.
Run `rlang::last_trace()` to see where the error occurred.`

When running get_last_report() this is the output:

$message
$message$statusCode
[1] 422

$message$error
[1] "Unprocessable Entity"

$message$messages
$message$messages[[1]]
$message$messages[[1]]$title
[1] "Wrong geometry size"

$message$messages[[1]]$detail
[1] "Geometry too large"

$status
[1] 422

I am using a shapefile of the Mediterranean IHO Sea Area from here https://www.marineregions.org/gazetteer.php?p=details&id=1905. Reading it in with sf::read_sf(). Now, when I use the GFCM RFMO, this does not produce any errors and I can use that area despite its size. Is this just a limitation in using a custom sf object or is there a way around this?

AndreaSanchezTapia commented 1 month ago

Hi @tom-lvn thanks for flagging this. Could you send the shapefile you're using here or to research [at] globalfishingwatch.org? I'll take a look or ping our API team. Thanks!

tom-lvn commented 1 month ago

Hey, I just sent an email with the shapefile, as I could not upload it here. Thanks already for your help!

AndreaSanchezTapia commented 1 month ago

Hi @tom-lvn , thanks for the patience while I came back to you.

We do have a limitation with the shapefiles, but more than size it's their complexity when we do a request. There are two main workarounds if you encounter this:

  1. If you don't care too much about the fine scale shapes of the shapefile, you can dissolve and simplify your shapefile, and redo the request.

  2. If the original shapefile is very important, you can send a bounding box of the shapefile. The API will return the dataset seamlessly, even if it's a larger area. Then you can crop the result locally to your desired shapefile. To get the bounding box:

bbox <- st_bbox(shapefile)  |>
  sf::st_as_sfc() |>
  sf::st_as_sf()

And send bbox in the USER_SHAPEFILE parameter.

A third option does not work in this case, you could loop over the polygons and do several requests, if the features were not complex themselves which is not the case here. In this case, the shapefile has 10 polygons and not all of them can fetch the information.

A third option would be to loop over the polygons. I received previously a 524 error for polygon 2 of the shapefile but redid the request and it worked. I will ask our API team about this, sometimes redoing the request just works.