search

16EAGLE / basemaps

A lightweight package for accessing basemaps from open sources in R 🗺️
https://jakob.schwalb-willmann.de/basemaps
GNU General Public License v3.0
58 stars 16 forks source link

caching tiles? #1

Open mdsumner opened 4 years ago

mdsumner commented 4 years ago

Hey there, this looks very nice - I wonder if you are interested in caching the tiles natively as ceramic does? In my mind ceramic is overloaded, it downloads tiles and (optionally) loads them as a raster, but really it should stick to download tiles for given service/extent, and have separated readers for those tile caches. There might be common ground here for us.

16EAGLE commented 4 years ago

Hi Michael, thanks – I am open to reduce redundancy and define common ground! One goal for basemaps was to serve as a dependency for moveVis (and other packages if needed) which is why I was aiming to use as little imports as possible and keep it lightweight.

In general, I am open to use ceramic for managing tile caching. What exactly do you mean with "natively as ceramic does" in contrast to how basemap caches tiles? basemaps currently caches both tiles (as PNGs) and cropped/reprojected products (as TIFS) so that running multiple queries with the same extents and projections just requires the package to read the finished TIF, while the underlying tiles as PNG would be recycled if a new extent or projection is queried.

Apart from that, there only are some functions in it to allow querying basemaps for objects crossing the dateline (e.g. visualizing unprojected movement trajectories that cross the dateline, something some users wanted to do with moveVis). It basically splits the extents at the dateline, pulls map tiles separately and merges them together, applying a coordinate system extended over the dateline towards the less-far reaching extent.

The rest actually is just the user-facing functions.

mdsumner commented 4 years ago

Ah I didn't think it kept the native tiles, I mean the zoom/x/y.png dirs - I'll have a closer look!

mdsumner commented 4 years ago

I see, I'll basically copy you and add osm and carto to ceramic (thanks!) - I think that'll be fairly easy, what you might be interested in is a scan of the ceramic cache for tiles that already exist. They look like this (it's changed as I'm updating to the new tiles/ scheme of mapbox).

If I run

ceramic::cc_location(ext, format = "png")

you see those nine tiles same as you would get with basemap i.e. 

tl <- ceramic_tiles(zoom = 13, type = "mapbox.satellite")
tl$fullname

..user/AppData/Local/Cache/.ceramic/api.mapbox.com/v4/mapbox.satellite/13/4346/2860.png 

..user/AppData/Local/Cache/.ceramic/api.mapbox.com/v4/mapbox.satellite/13/4346/2861.png 
...

but stored with the full source path, zoom and type (that's what I mean by native fwiw). So, there's no benefit for you except a possible saving of repeat downloads for users, and whatever other tile sets we could add (mapbox vector tiles is a good one I'm keen to get to).

I'll keep exploring how you've done this for good ideas I can steal ;) But, finally tile-caching is something I was obsessed with, maybe it's time is past and we don't need it anymore ... a problem I will have is invalidating the cache, when it would be better to get more recent tiles and so forth.

If I see easy opportunities for cross over I'll let you know.