consider geojsonsf

[SymbolixAU]

I realise this package is still in development, but I'd like to suggest using geojsonsf over geojsonio for speed

On the example data uol

library(microbenchmark)

uol <- rbind(geoplumber::uni_point, geoplumber::uni_poly)

microbenchmark(
  io = { 
    geoio <- geojsonio::geojson_json(uol)
  },
  sf = {
    geosf <- geojsonsf::sf_geojson(uol)
  }
)
# Unit: microseconds
# expr       min        lq       mean     median        uq      max neval
#  io 47649.880 55244.175 62266.4108 58047.6650 64066.635 130959.0   100
#  sf   281.001   291.102   637.7593   366.3335   407.578  16500.6   100
# There were 50 or more warnings (use warnings() to see the first 50)

on something bigger

nc <- sf::read_sf(system.file("shape/nc.shp", package="sf"))

microbenchmark(
    io = { 
        geoio <- geojsonio::geojson_json(nc)
    },
    sf = {
        geosf <- geojsonsf::sf_geojson(nc)
    },
    times = 5
)
# Unit: milliseconds
# expr        min         lq       mean     median         uq       max neval
#   io 510.809091 528.999507 620.537389 577.580768 692.760694 792.53688     5
#   sf   6.752821   7.005372   8.366773   7.107778   8.289001  12.67889     5

[Robinlovelace]

Thanks for the suggestion! In fact we have discussed alternatives and I ran a benchmarks showing geojsonio is no faster, if not slightly slower, than sf.

This geojsonsf** pkg seems blazing fast. Question: how is it so fast? For producing geojson representations of sf objects I imagine it's unbeatable. Great work, we'll consider it for sure.

[SymbolixAU]

Question: how is it so fast?

Everything's written in C++ / Rcpp, whereas geojsonio is all R.

For producing geojson representations of sf objects

This is exactly why I wrote it. I do a lot of work with spatial databases and needed a quick way to read & write.

[Robinlovelace]

Sounds good. Just noticed my previous comment was borked. Here's a link to the benchmark I ran with the same thing in mind: https://github.com/ATFutures/geoplumber/issues/27#issuecomment-421247231

I reckon we'll switch to geojsonsf. Thanks for sharing.

[layik]

Thank you @SymbolixAU. Thanks @Robinlovelace.

C++ is convincing, and will do.

[layik]

system.time(geojsonio::geojson_json(geoplumber::traffic))
#>    user  system elapsed 
#>   0.973   0.046   1.021
system.time(geojsonsf::sf_geojson(geoplumber::traffic))
#>    user  system elapsed 
#>   0.008   0.001   0.009

Created on 2018-09-16 by the reprex package (v0.2.0).

[layik]

# for now leaving the traffic data to be parsed every run.
time curl http://localhost:8000/api/data
# geojsonsf
real    0m0.046s
user    0m0.007s
sys 0m0.006s
#geojsonio
real    0m0.448s
user    0m0.008s
sys 0m0.013s

Committing.

[Robinlovelace]

Awesome fast work @layik. From kitchen surrounded by hung-over people. Showing the guests here the power of geographic data science!

[SymbolixAU]

If there's any features you want (such as adding an attr(x, "class") <- "json" or something let me know on my issues page

Also be aware of this one where elevation isn't supported yet

[layik]

Thanks for letting us know. I will watch out for the elevation. Right now, 2D is doing ok.

[layik]

25 @SymbolixAU just in case GitHub does not tag you in the other ticket. Big thank you!

[SymbolixAU]

@Robinlovelace @layik I've found another issue you should be aware of, and this one is a bit more serious

https://github.com/SymbolixAU/geojsonsf#warning-note-2018-09-22

I've submitted an update to rapidjsonr, so should be able to publish and update to geojsonsf in a day or so.

[layik]

Thanks for letting us know @SymbolixAU, will definitely watch out for it.

[SymbolixAU]

@Robinlovelace @layik So I can plan my updates to geojsonsf, what are your timelines for release of geoplumbr?

I've been working on a new JSON converter (jsonify) which, when complete, should make the conversion from sf to GeoJSON faster. But It won't be ready for a couple (2-4) weeks yet.

[Robinlovelace]

At least month of testing/feedback before we submit it to cran is my guess. Layik is the main person behind it tho so with getting his view 2.

[layik]

I just added two milestones, apologies to you both @SymbolixAU and @Robinlovelace been working on something else (very cool). So hopefully that gives us some rough timelines.

[SymbolixAU]

@layik thanks. I'll work to getting geojsonsf updated in mid-October.

[SymbolixAU]

Will it break any of your code if I make all the GeoJSON returned from sf_geojson() / sfc_geojson() have a "json" class attribute (it currently returns chracter)?

sfc_geojson(sf::st_sfc(sf::st_point(c(0,0))))
# [1] "{\"type\":\"Point\",\"coordinates\":[0.0,0.0]}"
# attr(,"class")
# [1] "json"

[SymbolixAU]

I've also resolved a lot of issues ready for v1.2.

Of note is is handling elevation (Z/M) attributes, dates, and speed improvement for converting sf -> GeoJSON.

If it breaks any of your tests please let me know. I'm aiming for a 31st October release.

[layik]

I should re-enable email notifications. Apologies @SymbolixAU for late reply. No, it won't.

I have patched current issue with adding an empty property to the df before converting it to GeoJSON. Awesome work RE new issues for v1.2!

Let me know if I can be of any other help.

[layik]

Dear @SymbolixAU, with both geojsonsf and now considering jsonify, I am hitting the block that I either use both geojsonio and jsonlite in parallel or I am struggling to do my reads both from disk and remote paths. Do you suggest a way forward for now? Use geojsonio for read and geojsonsf for conversion?

Thank you!

[SymbolixAU]

@layik I'm not sure I understand your question. Do you need a way to read a geojson file from disk?

[layik]

@SymbolixAU indeed or best, one that would take a remote URL or a local path, the equivalent of geojsonio::geojson_read() function.

[SymbolixAU]

geojson_sf() can read urls

from ?geojson_sf

Arguments geojson | string or vector of GeoJSON, or a URL or file pointing to a geojson file

[SymbolixAU]

i.e.

wpz <- geojsonsf::geojson_sf("http://opendata.canterburymaps.govt.nz/datasets/fb00b553120b4f2fac49aa76bc8d82aa_26.geojson")

works

[layik]

Oh! You guys need some docs work, and I apologize if I cannot send a PR. I did search for read!

[layik]

Testing local paths.

[layik]

Interesting! That gives me an sf data.frame class. I have unfinished code to do that and you guys have done it. Thank you.

[SymbolixAU]

There's just one thing missing, but I don't think it prevents the object from being valid sf

[layik]

Well, I will cross that bridge when I get there. No idea what it is. Thank you for being around!