Error in geo_raw[[i]]: subscript out of bounds

[ar-puuk]

Describe the bug I am unable to download a particular layer from a ArcGIS Mapserver. Every other layer from 0-8, 10-12 works, I only have this issue with the layer 9: WFRC RTP Active Transportation Projects.

To Reproduce

# Load required libraries
library(arcgislayers)

furl <- "https://gis.wfrc.org/arcgis/rest/services/UTAMap/UTA_FirstLast/MapServer"

conn <- arcgislayers::arc_open(furl)
conn
#> <MapServer <13 layers, 0 tables>>
#> CRS: 3857
#> Capabilities: Map,Query,Data
#>   0: Point Project Recommendations (esriGeometryPoint)
#>   1: Linear Project Recommendations (esriGeometryPolyline)
#>   2: TRAX Stations (esriGeometryPoint)
#>   3: FrontRunner Stations (esriGeometryPoint)
#>   4: FrontRunner (esriGeometryPolyline)
#>   5: TRAX (esriGeometryPolyline)
#>   6: Bus Stops (esriGeometryPoint)
#>   7: Existing Active Transportation Network (esriGeometryPolyline)
#>   8: MAG RTP Active Transportation Projects (esriGeometryPolyline)
#>   9: WFRC RTP Active Transportation Projects (esriGeometryPolyline)
#>   10: Bus Routes (esriGeometryPolyline)
#>   11: Commuter Rail Walkshed (esriGeometryPolygon)
#>   12: Light Rail Walkshed (esriGeometryPolygon)

layer <- arcgislayers::get_layer(conn, id = 9)
layer
#> <FeatureLayer>
#> Name: WFRC RTP Active Transportation Projects
#> Geometry Type: esriGeometryPolyline
#> CRS: 3857
#> Capabilities: Map,Query,Data

# Your original code
data <- arcgislayers::arc_select(layer)
#> Error in geo_raw[[i]]: subscript out of bounds

^{Created on 2024-03-11 with reprex v2.1.0}

Expected behavior The data was supposed to be loaded as other layers.

Additional context The layer works in ArcGIS Online

[elipousson]

I think this may actually be an issue with arcgisutils::parse_esri_json(). If I have a chance, I'll try to narrow down the source of the problem and open a new issue in the arcgisutils repository.

[JosiahParry]

The issue is a null geometry in OBJECTID 1481. A solution here is to use st_geomrtry_collection() in the presence of a null geometry. I'm out of the office as Esri DevSummit so I can make a fix yet! But if you include where = "objectid <> 1481" in the arc_select() statement.

It's on my roadmap to handle all geometry processing using Rust so we don't have this issue!

Note to self: have test for null geometries

[JosiahParry]

@ar-puuk if you install the dev version of {arcgisutils} via pak::pak("r-arcgis/arcgisutils") this should now work. Let me know!

[ar-puuk]

@JosiahParry Thanks a lot.

To provide you with context, I am trying to download all layers from the Map server into a geodatabase using the sf package. When I had issues with a layer with id = 9, I isolated the issue to that particular layer and simplified the code for reprex. This reprex chunk of code now works with the updated arcgisutils.

However, I am still having the same issue with my initial code I began with. I tried loading arcgislayer and arcgisutils in a different order but that didn't help. The layers until the layer id = 8 works, but the whole thing fails once the iteration reaches id = 9. Since I am unable to isolate the problem this time, I am attaching the whole code. Please ignore the sf warnings.

library(arcgislayers)
library(arcgisutils)
library(sf)
#> Linking to GEOS 3.11.2, GDAL 3.7.2, PROJ 9.3.0; sf_use_s2() is TRUE

# Set Output Folder
wd = "D:/"

# setup the name of the Geodatabase
gdb_name <- paste0(as.character(format(Sys.Date(), "%Y.%m.%d")), "_UTA_FirstLast.gdb")

# Set URL of ARCGIS Server
server <- "https://gis.wfrc.org/arcgis/rest/services/UTAMap/UTA_FirstLast/MapServer" # UTA First Last Mile

# Open connection to remote resource
conn <- arcgislayers::arc_open(server)

for (i in seq_along(conn$layers)) {

  # Extract name of the layer
  name = conn$layers$name[i]

  cat(i - 1, ": ")

  # Use tryCatch to handle errors during layer extraction and selection
  tryCatch({
    # Extract a layer from a Feature or Map Server
    layer <- arcgislayers::get_layer(conn, name = name)

    cat(name, ": ")

    # Query a Feature Service
    data <- arcgislayers::arc_select(layer)

    cat("Complete!\n")

    # Export data
    sf::write_sf(data, file.path(wd, gdb_name), layer = name, append = FALSE)
  }, error = function(e) {
    cat(conditionMessage(e), "\n")
  })
}
#> 0 : Point Project Recommendations : Complete!
#> Warning in CPL_write_ogr(obj, dsn, layer, driver,
#> as.character(dataset_options), : GDAL Message 6: Normalized/laundered layer
#> name: 'Point Project Recommendations' to 'Point_Project_Recommendations'
#> 1 : Linear Project Recommendations : Complete!
#> Warning in CPL_write_ogr(obj, dsn, layer, driver,
#> as.character(dataset_options), : GDAL Message 6: Normalized/laundered layer
#> name: 'Linear Project Recommendations' to 'Linear_Project_Recommendations'
#> 2 : TRAX Stations : Complete!
#> Warning in CPL_write_ogr(obj, dsn, layer, driver,
#> as.character(dataset_options), : GDAL Message 6: Normalized/laundered layer
#> name: 'TRAX Stations' to 'TRAX_Stations'
#> 3 : FrontRunner Stations : Complete!
#> Warning in CPL_write_ogr(obj, dsn, layer, driver,
#> as.character(dataset_options), : GDAL Message 6: Normalized/laundered layer
#> name: 'FrontRunner Stations' to 'FrontRunner_Stations'
#> 4 : FrontRunner : Complete!
#> 5 : TRAX : Complete!
#> 6 : Bus Stops : Complete!
#> Warning in CPL_write_ogr(obj, dsn, layer, driver,
#> as.character(dataset_options), : GDAL Message 6: Normalized/laundered layer
#> name: 'Bus Stops' to 'Bus_Stops'
#> 7 : Existing Active Transportation Network : Complete!
#> Warning in CPL_write_ogr(obj, dsn, layer, driver,
#> as.character(dataset_options), : GDAL Message 6: Normalized/laundered layer
#> name: 'Existing Active Transportation Network' to
#> 'Existing_Active_Transportation_Network'
#> 8 : MAG RTP Active Transportation Projects : Complete!
#> Warning in CPL_write_ogr(obj, dsn, layer, driver,
#> as.character(dataset_options), : GDAL Message 6: Normalized/laundered layer
#> name: 'MAG RTP Active Transportation Projects' to
#> 'MAG_RTP_Active_Transportation_Projects'

^{Created on 2024-03-15 with reprex v2.1.0}

[JosiahParry]

Please reinstall arcgisutils using pak::pak("r-arcgis/arcgisutils"). You have to reinstall the package and restart your session for the new package to be recognized.

I can read in all of the data from the MapServer using the below without a problem now.

library(arcgislayers)

furl <- "https://gis.wfrc.org/arcgis/rest/services/UTAMap/UTA_FirstLast/MapServer"

conn <- arcgislayers::arc_open(furl)

layers <- get_all_layers(conn)

res <- lapply(layers[[1]], arc_select)

[ar-puuk]

I apologize. That was an error on my part. Your code works even better for my purpose. Thank you so much for addressing my issue. Much appreciated!