Create imagery tile service

[nvkelso]

We've talked for a while about creating a "satellite" / aerial imagery service. This issue is a placeholder for that.

Pros:

• Last big hole in the Mapzen mapping platform (now that traffic is underway)
• Mapquest Open provided this service but shut down in 2016. USGS service will shut down soon, too.
• There is a wealth of public domain or CC-BY data in the USA and a few other countries globally that would be good for street level detail. The United States is about 50% of Mapzen's tile market.
• Amazon offers Landsat (global), Setinel-2 (global), and NAIP (USA) as public data on AWS.

Cons:

• Paying for satellite imagery is against Mapzen's open data policy.
• Much of Europe doesn't have street-level imagery via open imagery sources. This is about 50% of Mapzen's tile market.
• Rest of world doesn't have good open imagery sources past Landsat

See also:

• http://devblog.mapquest.com/2016/06/15/modernization-of-mapquest-results-in-changes-to-open-tile-access/
• http://www.directionsmag.com/pressreleases/usgs-cutting-high-resolution-ortho-imagery-program/479365
• https://aws.amazon.com/public-datasets/naip/
• https://aws.amazon.com/public-datasets/sentinel-2/
• https://pages.awscloud.com/public-data-sets-landsat.html

[iandees]

Related: I've been thinking about building a basic imagery layer for OSM US ever since the USGS broke their publicly-hosted Esri server: https://github.com/osmlab/editor-layer-index/issues/151

[iandees]

Where "basic" means: Sentinel 2/Landsat worldwide, NAIP United States-wide, USGS High-Res Orthoimagery (HRO) for some parts of the US. After speaking with USGS, this is about 100TB of raw imagery for the HRO and NAIP pieces.

I've also uncovered tons of localities that post more recent and higher-resolution imagery on their open data portals or host it on their Esri servers. This data could eventually be included in such a layer as well.

Most importantly: I wouldn't want to spend the time (at first) doing the color matching and whatnot for a perfect mosaic. I would want to set that expectation up front and seek people from the community to help with that if it's important to them.

Seth is also interested in this.

[nvkelso]

+1, and Seth's work around OpenAerialMap seems to have laid good groundwork for this. HOT and RedCross (?) are mostly focused small area imagery from drones, balloons, and little airplane flights. But there's a good process for taking in new data, previewing them on a slippy map, mosiacing with other sources, and merging into general imagery endpoint.

Update on NAIP and HRO: those are available via Amazon public data they but under the "experimental" requester-pays model and also prevents GDAL seek access. Probably better to re-get them from USGS :\

[zerebubuth]

Some random thoughts:

I think @iandees is right about starting with "basic". It's what we did with terrain and that seems to have been "good enough" to start with and cause some excitement, despite some notable deficiencies outside the US. I think it also encouraged a lot of people to come forward with additional datasets. ("Start where you are"?)

The presence of an open data project, I think, makes it more likely that agencies with data will open it up. I've seen this with OSM, and I'm fairly sure it's happening with OpenAddresses and Transit.land as well. I would have high hopes that a credible (and working) successor to OpenAerial would prompt similar openness.

Imagery may end up being quite different from DEM data, and I think we should have a think about how we'd like to handle it. With Joerd, we generated all tiles globally down to z15 (~4.8m/pix). That's enough to capture most DEMs, which seem to be in the 3-30m range. Although NED is pretty exceptional in being 1m in some places, and we had trouble capturing that.

NAIP appears to be 1-2m (~z17), and I've been noticing that 13cm imagery is becoming more common (~z20). Landsat, on the other hand, is 30-60m (~z12). I don't think we'd want to generate the whole world down to z17, and I'm not sure it would be possible to do down to z20 (would be $5.5 million just for the PUTs to S3).

So, getting to the point finally, I think we need to figure out:

1. If we're going to pre-generate, how to handle (and advertise) different resolutions. We could mask at z12 and go down to z20 everywhere else, although that would be more tiles than we need to sample NAIP by a factor of 85.
2. If we're not going to pre-generate (or pre-generate down to z12) how do we structure the computation to be efficient? For example, it might be worth pre-generating z12 tiles, projected and at maximum resolution per-dataset, and then down-sampling and cropping them on a per-request basis.
3. Joerd doesn't have a good system for updates, which makes it difficult to pull in new datasets. I think that is a barrier to contribution, and we should address that for imagery. Concerns around updating interact a lot with decisions about how much to pre-generate.