Serving MBTiles for Mapbox Studio

[lukasmartinelli]

Tessera is an amazing piece of work!! Great how you managed to make it so easy to run a server for different tilelive-sources.

What I would like to use Tessera for is to serve an mbtiles file to Mapbox Studio (because it can only work with TileJSON urls no mbtiles).

When specifying the TileJSON endpoint in Mapbox Studio something is not right with the X-Y axis or projection. The map is like tiled into several pieces.

Reproduce:

Using the following MBTiles. https://github.com/klokantech/vector-tiles-sample/releases/download/v1.0/countries.mbtiles

tessera -p 80 mbtiles://./countries.mbtiles

Visiting localhost:80 everything works fine.

Clone repo https://github.com/klokantech/vector-tiles-sample/ If we open the project and swap out the source with the TileJSON source it starts getting weird.

Now if we look at the debug view inside the studio it look like the x tiles are at the wrong location.

[lukasmartinelli]

It works with tileserver-php. Perhaps it is also Mapbox Studio that does not support TMS? It would be really cool if tessera and Mapbox Studio could function together.

Produced TileJSON with thileserver-php:

{
   "attribution":"Natural Earth Data",
   "center":"0,7.0137,0",
   "description":"Natural Earth Data with .shp data from  TileMill",
   "format":"pbf",
   "maxzoom":6,
   "minzoom":0,
   "name":"Countries",
   "bounds":[
      -180,
      -85.738076382392,
      180,
      84.79842793857
   ],
   "profile":"mercator",
   "basename":"countries",
   "tilejson":"2.0.0",
   "scheme":"xyz",
   "tiles":[
      "http://192.168.99.100/countries/{z}/{x}/{y}.pbf"
   ],
   "grids":[
      "http://192.168.99.100/countries/{z}/{x}/{y}.grid.json"
   ],
   "vector_layers":[ /* vector layers */ ]
}

Produced TileJSON with tessera:

{
   "scheme":"tms",
   "basename":"countries.mbtiles",
   "id":"countries",
   "filesize":9376768,
   "attribution":"Natural Earth Data",
   "center":[
      0,
      7.0137,
      0
   ],
   "description":"Natural Earth Data with .shp data from  TileMill",
   "format":"pbf",
   "maxzoom":6,
   "minzoom":0,
   "name":"Countries",
   "bounds":[
      -180,
      -85.05112877980659,
      180,
      85.0511287798066
   ],
   "tiles":[
      "http://localhost/{z}/{x}/{y}.pbf"
   ],
   "tilejson":"2.0.0",
   "vector_layers":[ /* vector layers */ ]
}

[klokan]

A running URL from the TileServer.php project for testing: http://tileserver.maptiler.com/countries.json

The issue is the difference between XYZ and TMS, so scheme: tms is the problem. Either it is necessary to remove from tilejson this formula and/or make conversion on the server side for serving the tiles on z/x/y, where:

y =  (2^z - 1) - y

[mojodna]

@klokan is right on about TMS vs. XYZ schemes. Either the MBTiles is wrong, mbtiles is reporting the wrong scheme, or tessera is lying. I'm taking a deeper look now.

I've been swamped, so apologies for the delay.

[mojodna]

npm install tessera tilelive-xray mbtiles
curl -LO https://github.com/klokantech/vector-tiles-sample/releases/download/v1.0/countries.mbtiles
node_modules/.bin/tessera xray+mbtiles://./countries.mbtiles

x-ray TileJSON (note: no scheme):

$ curl -s http://localhost:8080/index.json | jq .
{
  "bounds": [
    -180,
    -85.0511,
    180,
    85.0511
  ],
  "center": [
    0,
    0,
    3
  ],
  "format": "png8:m=h",
  "maxzoom": 22,
  "minzoom": 0,
  "scale": "1",
  "name": "Untitled",
  "tiles": [
    "http://localhost:8080/{z}/{x}/{y}.png"
  ],
  "tilejson": "2.0.0"
}

MBTiles TileJSON (tms scheme):

$ node_modules/.bin/tessera mbtiles://./countries.mbtiles &
Listening at http://0.0.0.0:8080/
$ curl -s http://localhost:8080/index.json | jq .
{
  "scheme": "tms",
  "basename": "countries.mbtiles",
  "id": "countries",
  "filesize": 9376768,
  "attribution": "Natural Earth Data",
  "center": [
    0,
    7.0137,
    0
  ],
  "description": "Natural Earth Data with .shp data from  TileMill",
  "format": "pbf",
  "maxzoom": 6,
  "minzoom": 0,
  "name": "Countries",
  "vector_layers": [
    {
      "id": "state",
      "description": "",
      "fields": {
        "NAME1": "String",
        "SCALERANK": "Number",
        "COUNTRYNAM": "String",
        "FeatureCla": "String",
        "ADM0_A3": "String",
        "MAP_COLOR": "Number"
      }
    },
    {
      "id": "country",
      "description": "",
      "fields": {
        "ScaleRank": "Number",
        "LabelRank": "Number",
        "FeatureCla": "String",
        "SOVEREIGNT": "String",
        "SOV_A3": "String",
        "ADM0_DIF": "Number",
        "LEVEL": "Number",
        "TYPE": "String",
        "ADMIN": "String",
        "ADM0_A3": "String",
        "GEOU_DIF": "Number",
        "NAME": "String",
        "ABBREV": "String",
        "POSTAL": "String",
        "NAME_FORMA": "String",
        "TERR_": "String",
        "NAME_SORT": "String",
        "MAP_COLOR": "Number",
        "POP_EST": "Number",
        "GDP_MD_EST": "Number",
        "FIPS_10_": "Number",
        "ISO_A2": "String",
        "ISO_A3": "String",
        "ISO_N3": "Number"
      }
    },
    {
      "id": "land-border-country",
      "description": "",
      "fields": {
        "ScaleRank": "Number",
        "FeatureCla": "String"
      }
    },
    {
      "id": "geo-lines",
      "description": "",
      "fields": {
        "DISPLAY": "String",
        "ScaleRank": "Number"
      }
    },
    {
      "id": "country-name",
      "description": "",
      "fields": {
        "ScaleRank": "Number",
        "LabelRank": "Number",
        "Z_POSTAL": "Number",
        "Z_ABBREV": "Number",
        "Z_NAME": "Number",
        "Z_ADMIN": "Number",
        "FeatureCla": "String",
        "SOVEREIGNT": "String",
        "SOV_A3": "String",
        "ADM0_DIF": "Number",
        "LEVEL": "Number",
        "TYPE": "String",
        "ADMIN": "String",
        "ADM0_A3": "String",
        "GEOU_DIF": "Number",
        "NAME": "String",
        "ABBREV": "String",
        "POSTAL": "String",
        "NAME_FORMA": "String",
        "TERR_": "String",
        "NAME_SORT": "String",
        "MAP_COLOR": "Number",
        "POP_EST": "Number",
        "GDP_MD_EST": "Number",
        "FIPS_10_": "Number",
        "ISO_A2": "String",
        "ISO_A3": "String",
        "ISO_N3": "Number"
      }
    }
  ],
  "bounds": [
    -180,
    -85.05112877980659,
    180,
    85.0511287798066
  ],
  "tiles": [
    "http://localhost:8080/{z}/{x}/{y}.pbf"
  ],
  "tilejson": "2.0.0"
}

[mojodna]

(Still digging, updating my copy of Mapbox Studio Classic)

[mojodna]

Reproduced with Mapbox Studio. Haven't figured out what's lying yet.

Workaround: create a non-dynamic copy of index.json with a manually modified scheme and use that as your TileJSON:

curl -s http://localhost:8080/index.json | jq . | sed 's/tms/xyz/' > countries.json

[mojodna]

Editing project.yml (in the tm2 project directory) and changing the source to mbtiles:///path/to/countries.mbtiles works:

npm install tessera tilelive-tmstyle mbtiles
node_modules/.bin/tessera tmstyle:///path/to/project.tm2

[mojodna]

Reproduced with tilelive-tmstyle (tilelive-vector under the hood) using an HTTP source (tilelive-http delegating to tilejson, which is also what Mapbox Studio uses for HTTP-accessible tiles).

This makes me think that tilejson may be the culprit, as that's the only difference in the pipeline between HTTP + direct MBTiles.

[mojodna]

Yup. tilejson is the culprit, as it transparently transforms the coordinates from TMS to XYZ (https://github.com/mapbox/node-tilejson/blob/master/lib/tilejson.js#L170) without modifying the reported scheme (https://github.com/mapbox/node-tilejson/blob/master/lib/tilejson.js#L137-L140 ; data isn't modified elsewhere).

I'll put a patch together and submit it to Mapbox.

[mojodna]

https://github.com/mapbox/node-tilejson/pull/42

[lukasmartinelli]

I'll put a patch together and submit it to Mapbox.

You're the best!! Thanks for digging down the tilelive stack.