PMTiles hosted via CloudFront/S3

[thisisaaronland]

This:

https://github.com/sfomuseum/go-http-protomaps

Comes with an example application that contains an embedded pmtiles and works fine when run and served like this:

> go run -mod vendor cmd/example/main.go
2021/04/27 16:34:36 Listening for requests on http://localhost:8080

If I tell the example server to fetch pmtiles data from a remote CloudFront/S3 server however there are no JS errors but nothing is ever rendered. For example:

> go run -mod vendor cmd/example/main.go -protomaps-tile-url https://static.sfomuseum.org/pmtiles/sfo.pmtiles
2021/04/27 16:36:19 Listening for requests on http://localhost:8080

Looking at the network console I see a whole bunch of OPTIONS and HEAD requests but then they just stop. Eventually I noticed that I need to include the allow_200 option but am I missing something else?

[bdon]

I can reproduce this, and it definitely seems like something that should "just work" - need to dig more into this.

Wanted to clarify what allow_200 means: some CDNs and Storage APIs, such as Azure Storage and GitHub Pages, will return HTTP status code 200 for a HEAD request with a Range: header instead of code 206. This means the server behavior is indistinguishable from an HTTP server that doesn't support Byte Serving at all like the Python 3 http.server toy implementation.

we don't want to accidentally slurp huge files from a server that doesn't support byte serving; we also can't rely on the accept-range header since this isn't readable from JavaScript.

I would prefer that CDNs reply consistently to a HEAD+ Range: with 206; not enough of an HTTP expert to be able to argue that is it out of spec. Until then my bandaid solution is to have the code author explicitly indicate allow_200 to work with those CDNs.

[bdon]

@thisisaaronland turns out this is the same core issue as https://github.com/protomaps/protomaps.js/issues/6#issuecomment-826533159 : you also need to pass allow_200 to the LeafletLayer creation like this, because to get the metadata in the example I create a totally separate PMTiles instance:

var layer = new protomaps.LeafletLayer({url:'YOUR_FILE.pmtiles',allow_200:true})

It turns out that CloudFront returns the 200 instead of 206 like I described (and I think is out of spec). I realize this is a huge gotcha right now and we should consider if just allowing 200 is okay, and find some other way to warn the user about a server that doesn't support byte serving

[bdon]

Related issue: https://github.com/protomaps/PMTiles/issues/11

[thisisaaronland]

Perfect, thanks!

https://github.com/sfomuseum/go-http-protomaps/commit/73a04efa95828901b678646a51f0199e6f31644f

[bdon]

@thisisaaronland with protomaps.js version 0.1.0 you don't need to include the pmtiles script anymore, nor do you need allow_200:

const p = new protomaps.PMTiles(tile_url);

p.metadata().then(m => {
   let bounds_str = m.bounds.split(',')
   let bounds = [[+bounds_str[1],+bounds_str[0]],[+bounds_str[3],+bounds_str[2]]]
   layer = new protomaps.LeafletLayer({url:p, bounds:bounds})
   layer.addTo(map)
   map.fitBounds(bounds)
});

[thisisaaronland]

Thanks, I've updated sfomuseum/go-http-protomaps to bundle v0.1.0 versions of the necessary JS files. I noticed a couple of things:

• I still need to load pmtiles.js - without it I get "pmtiles not defined" errors when calling new pmtiles.PMTiles
• This declaration makes JS sad: p.metadata().then(m => { - I switched back to the original p.metadata(m => { syntax and everything worked again.

Maybe this is caching issue in unpkg.com? Would it be possible to include bundled/compiled version in the GitHub repo itself, in a dist folder or equivalent?

[bdon]

@thisisaaronland I goofed, it should be protomaps.PMTiles (the embedded version of the library) and not pmtiles.PMTiles.

[thisisaaronland]

Fixed and works, thanks.

https://github.com/sfomuseum/go-http-protomaps/commit/5559e9487519df0e7cad92d8e09ce6336b24197f