ATFutures / geoplumber

Serve geographic data from R and consume with scalable front end.
https://atfutures.github.io/geoplumber/
59 stars 7 forks source link

Make gp_plumb use sf object by default #27

Closed Robinlovelace closed 6 years ago

layik commented 6 years ago

Above push introduces necessary dependancy @Robinlovelace.

Two solutions:

  1. further process uni_poly and uni_point and avoid dependancy.
  2. I am happy with geojsonio, if there is an alternative, happy to listen but might not be necessary to add custom function.

If you have a preference, will sort it out.

Robinlovelace commented 6 years ago

Go with what works, also happy with importing geojsonio + sf.

I think those two packages are very useful for geo* stuff so expected. Happy with another npm dep also.

One question: what can geojsonio do that sf cannot (as you know I'm a big fan of that package but there are other options this.

layik commented 6 years ago

I am a fan of it too!

Does it convert an sf object/df into geojson (not json)? I would use sf if it did.

https://github.com/ropensci/geojsonio/blob/master/R/geojson_json.R

layik commented 6 years ago

It is using jsonlite to do the rest

https://github.com/ropensci/geojsonio/blob/3c4379200c693485a8c0534abb6db8b26fa7bba5/R/zzz.r#L12

Robinlovelace commented 6 years ago

@layik it can save an sf object as a .geojson file but maybe not convert it in memory. Mini test below:

devtools::install_github("ATFutures/geoplumber")
#> Using GitHub PAT from envvar GITHUB_PAT
#> Downloading GitHub repo ATFutures/geoplumber@master
#> from URL https://api.github.com/repos/ATFutures/geoplumber/zipball/master
#> Installing geoplumber
#> '/usr/lib/R/bin/R' --no-site-file --no-environ --no-save --no-restore  \
#>   --quiet CMD INSTALL  \
#>   '/tmp/Rtmp3TGHxJ/devtools2b47ceb83cf/ATFutures-geoplumber-fef08ed98e29264f55cb4a52726a09a7834a5133'  \
#>   --library='/home/robin/R/x86_64-pc-linux-gnu-library/3.5'  \
#>   --install-tests
#> 

# get uni files as geojson
g1 = geojsonio::geojson_json(geoplumber::uni_point)
unclass(g1)
#> [1] "{\"type\":\"FeatureCollection\",\"features\":[{\"type\":\"Feature\",\"properties\":{\"osm_id\":\"84656164\",\"name\":\"University of Leeds\",\"amenity\":\"university\",\"note\":\"Not all owned by uni, this area is misleading\"},\"geometry\":{\"type\":\"Point\",\"coordinates\":[-1.5562643,53.80631]}}]}"
#> attr(,"type")
#> [1] "FeatureCollection"
#> attr(,"no_features")
#> [1] "1"
#> attr(,"five_feats")
#> [1] "Point"
st_json = function(x, f = "temp.geojson") {
  sf::st_write(x, f)
  readLines(f)
  file.remove("temp.geojson")
}

g2 = st_json(geoplumber::uni_point)
#> Writing layer `temp' to data source `temp.geojson' using driver `GeoJSON'
#> features:       1
#> fields:         4
#> geometry type:  Point

devtools::install_github("r-lib/bench")
#> Using GitHub PAT from envvar GITHUB_PAT
#> Skipping install of 'bench' from a github remote, the SHA1 (b59f722f) has not changed since last install.
#>   Use `force = TRUE` to force installation
bench::mark(iterations = 1,
  g2 = st_json(geoplumber::uni_point)
)
#> Writing layer `temp' to data source `temp.geojson' using driver `GeoJSON'
#> features:       1
#> fields:         4
#> geometry type:  Point
#> Writing layer `temp' to data source `temp.geojson' using driver `GeoJSON'
#> features:       1
#> fields:         4
#> geometry type:  Point
#> Warning: Some expressions had a GC in every iteration; so filtering is
#> disabled.
#> # A tibble: 1 x 10
#>   expression    min   mean median    max `itr/sec` mem_alloc  n_gc n_itr
#>   <chr>      <bch:> <bch:> <bch:> <bch:>     <dbl> <bch:byt> <dbl> <int>
#> 1 g2         10.2ms 10.2ms 10.2ms 10.2ms      98.2    4.02MB     1     1
#> # ... with 1 more variable: total_time <bch:tm>
bench::mark(iterations = 1,
  g1 = geojsonio::geojson_json(geoplumber::uni_point)
)
#> # A tibble: 1 x 10
#>   expression    min   mean median    max `itr/sec` mem_alloc  n_gc n_itr
#>   <chr>      <bch:> <bch:> <bch:> <bch:>     <dbl> <bch:byt> <dbl> <int>
#> 1 g1         17.6ms 17.6ms 17.6ms 17.6ms      56.7        0B     0     1
#> # ... with 1 more variable: total_time <bch:tm>

# about the same runtime

Created on 2018-09-14 by the reprex package (v0.2.0).

layik commented 6 years ago

Great to know. Verdict?

Robinlovelace commented 6 years ago

Both are fast. Both are fine. Use both if it's easier with geojsonio. Use just sf if possible with no overheads. But geojsonio is fine and plugs into other things that may be useful: https://github.com/ropensci?utf8=%E2%9C%93&q=geo&type=&language=

layik commented 6 years ago

Thanks. In this case writing to disk will cost high.

So for now I will add geojsonio as a dependancy.

layik commented 6 years ago

Let me use this ticket for something else.

People would want to remove serving the two objects at some point.

Above push is radical. Needs to move to the Inst/plumber.R

Robinlovelace commented 6 years ago

Yep - avoiding disk I/O is a big benefit of geojsonio. Not 100% sure what you mean by the 'something else' but sure you're on it. I'm heading off down south now so will pick-up Monday. Have a great (long) weekend and good luck with the open sourcing. Ready from my perspective but your call when it's good to :ship:

layik commented 6 years ago

Enjoy the party! (long) weekend? :)

layik commented 6 years ago

'plumb'ed before build? screenshot from 2018-09-14 10-33-57

after build! screenshot from 2018-09-14 10-37-11

Robinlovelace commented 6 years ago

Looks good! And to re-download the .geojson? One for the readme I think, that is really cool: geojson -> leaflet/download.