Dealing with invalid GeoJSON

[sckott]

arose from https://stackoverflow.com/questions/50240935/removing-brackets-from-ends-of-geojson-in-r

With the bad.txt file below, which contains JSON, rgdal fails without a helpful message:

library(rgdal)
rgdal::readOGR("bad.txt", rgdal::ogrListLayers("bad.txt"))
#> Error in rgdal::ogrListLayers("bad.txt") : Cannot open data source
# sf similarly doesn't give a helpful error
sf::st_read("bad.txt")
#> Cannot open data source /Users/sckott/github/ropensci/geojsonio/bad.txt
#> Error in CPL_read_ogr(dsn, layer, as.character(options), quiet, type,  :
#>   Open failed.

With good.txt it works fine:

rgdal::readOGR("good.txt", rgdal::ogrListLayers("good.txt"))
#> OGR data source with driver: GeoJSON
#> Source: "/Users/sckott/github/ropensci/geojsonio/good.txt", layer: "OGRGeoJSON"
#> with 1 features
#> It has 51 fields
#> An object of class "SpatialPolygonsDataFrame"
#> Slot "data":
#> ...
# sf works fine with good data
sf::st_read("good.txt")

Options:

• Use our pkg geojsonlint to lint json to attempt to make sure the json is valid geojson
• don't do anything - check on a case by case basis as users report errors? and could document that users should check that their GeoJSON is valid.
• any others?

If we use geojsonlint

# good (should all be TRUE, can check to see why hint is not working)
geojsonlint::geojson_hint(as.location('good.txt'))
#> [1] FALSE
geojsonlint::geojson_lint(as.location('good.txt'))
#> [1] TRUE
geojsonlint::geojson_validate(as.location('good.txt'))
#> [1] TRUE

# bad (should all be FALSE)
geojsonlint::geojson_hint(as.location('bad.txt'))
#> [1] FALSE
geojsonlint::geojson_lint(as.location('bad.txt'))
#> [1] FALSE
geojsonlint::geojson_validate(as.location('bad.txt'))
#> [1] FALSE

bad bad.txt good good.txt

[sckott]

cc @ateucher

[sckott]

moving to next milestone - played around with this a bit, but it's complicated since at least with geosjon_read a user can pass in files and urls, each with various extensions, so makes it quite complicated. e.g. we'd not want to lint a kml file or a shp file

here's what i tried:

#' @export
geojson_read.character <- function(x, method = "web", parse = FALSE, what = "list", ...) { 
  lint_geojson(x)
  read_json(as.location(x), method, parse, what, ...)
}

lint_geojson <- function(x) {
  if (inherits(x, "character")) {
    if (file.exists(x)) {
      geojsonlint::geojson_validate(unclass(x), error = TRUE)
    }
  }
  if (inherits(x, "location_")) {
    if (attr(x, "type") == "file") {
      geojsonlint::geojson_validate(unclass(x), error = TRUE)
    }
  }
}

#' @export
geojson_read.location_ <- function(x, method = "web", parse = FALSE, what = "list", ...) {
  lint_geojson(x)
  read_json(x, method, parse, what, ...)
}