Create a batch job to generate static TMS tile sets in S3 from source rasters

[jwalgran]

We have decided that we do not need any runtime customization when we display land cover or soil rasters on the map, so we can save on redundant processing and simplify the web app by generating static tile sets.

Added to the skeleton project created in #11

The input should be a raster, and extent, a set of zoom levels to render, and an S3 bucket+directory destination. The output should be a set of PNGs like soils/z/x/y.png

[caseycesari]

A method I've used in the past for this:

• Use TileMill to style the data.
• Export a mbtiles tile set.
• Explode the tileset using mb-util.
• Upload the tiles to S3.

Here is a tool that wraps up those last couple steps: https://github.com/MinnPost/mbtiles2s3

[caseycesari]

Also, after the initial styling, you don't need to open the TileMill desktop application to generate the tiles. You can hit it from the command line. Here's an example of that.

[caseycesari]

Just realized that the issue references using the raster versions of these datasets. The above workflows I highlighted for vector datasets, so it may not be useful here.

[mmcfarland]

Still awaiting the full list of "display rasters" from Stroud, but there will be at least:

• NLCD
• Soils
• Lithology
• Elevation products (slope, contours/topology, flow direction, flow accumulation)

and possibly some showing density of:

• Agriculture animals
• Septic systems

So in other words, more than a few, most available at national scale. Still wondering where the inflection point is between storage and dynamically tiling + cache. I imagine cost will be the deciding factor (pre baking the tiles will always be less complex and easier to serve).

[caseycesari]

If a layer is available as a vector (soils, contours), do we prefer that for display purposes?

[mmcfarland]

Not sure - my thinking was if we need the rasters for calculation, we could have them do double duty for display. A lot of the DEM derived products will be delivered as raster and it seemed like there were few straight up vector sources - that actually may be changing.

[jwalgran]

There is a WMS service (via AGS) for the NLCD. Not sure if it has enough resolution and performance for our display needs.

http://raster.nationalmap.gov/arcgis/services/LandCover/USGS_EROS_LandCover_NLCD/MapServer/WmsServer?SERVICE=WMS&VERSION=1.3.0&REQUEST=GetMap&BBOX=38.71860054666684903,-102.45059108509752832,38.94218062349745679,-102.06386800029764572&CRS=EPSG:4326&WIDTH=1044&HEIGHT=603&LAYERS=30&STYLES=&FORMAT=image/png&DPI=72&MAP_RESOLUTION=72&FORMAT_OPTIONS=dpi:72

[jwalgran]

I pulled the NLCD up on the viewer. It seems to perform well and it looks like the cells are 30m

[mmcfarland]

Using data-hub issues to track tiling requests.