syncpoint / openvtpk

Repackaging ESRI's VTPK into an MBTiles container
MIT License
51 stars 10 forks source link

field information #16

Open MannusEtten opened 2 years ago

MannusEtten commented 2 years ago

the mbtiles file created from my vtpk doesn't have any field information: {"vector_layers": [{"id":"bouwtekening","fields":{}}]}

so the symbology/style is also not working

ThomasHalwax commented 2 years ago

Hi @MannusEtten,

thank you for your feedback. Would it be possible to share your VTPK in order to debug the issue?

MannusEtten commented 2 years ago

bouwtekening.zip inside the zip you will find it, the style is also included inside the vtpk

MannusEtten commented 2 years ago

@ThomasHalwax no pressure, but have you been able to look into the zipfile and the vtpk?

ThomasHalwax commented 2 years ago

@MannusEtten The value {} for fields is hard-coded in the current version. I had a look at your example file and the file metadata.json (folder "p12") does not contain any values for fields either:

"vector_layers": [
    {
      "id": "bouwtekening",
      "description": "bouwtekening",
      "minzoom": 0,
      "maxzoom": 21,
      "fields": {}
    }
  ]

The examples provided by @spatialillusions TPK2GPKG do not even contain a metadata.json file.

Since I need to reverse engineer the VTPK specification it's pretty hard to find the correct way to extract the field information.

The debug output of the transform command shows

VectorTileLayer {
  version: 2,
  name: 'bouwtekening',
  extent: 536870912,
  length: 22,
  _pbf: {
    buf: Buffer(31277) [Uint8Array] [
       26, 169, 244,   1,  10,  12,  98, 111, 117, 119, 116, 101,
      107, 101, 110, 105, 110, 103,  18,  64,  18,   2,   0,   0,
       24,   2,  34,  56,   9, 232, 184, 254, 134,   2, 158, 195,
      255, 169,   1,  10, 203,   2, 174,   2,   9, 220,   2, 153,
        2,  10, 201,   2, 172,   2,   9, 228,   7, 168,  13,  10,
      207,   2, 165,   2,   9, 232,  10, 169,  13,  10, 229,   1,
      181,   1,   9, 180,   7, 208,  11,  10, 133,   2, 234,   2,
       18, 186,   6,  18,   2,   0,   1,  24,   2,  34, 177,   6,
        9, 170, 153, 254,
      ... 31177 more items
    ],
    pos: 31277,
    type: 0,
    length: 31277
  },
  _keys: [ '_symbol', '_minzoom' ],
  _values: [
    11, 15, 16, 10,  7,  6, 18,
     0, 19,  8, 13, 14,  3,  5,
     2,  1, 17,  4,  9, 12, 20,
    21
  ],
  _features: [
       19,    85,   914,  2846,
     2878,  6831, 10641, 11043,
    11199, 11236, 14232, 14481,
    16180, 16728, 18014, 18192,
    18275, 23691, 29310, 30642,
    30845, 31114
  ]
}

Here we can find _keys that seem to contain the value '_symbol' as your metadata.json does. But what about _minzoom? And how should I guess the type of the field? Maybe I can do that by looking at the _values field.

In order to make a solid decision I would need more complex examples.

spatialillusions commented 2 years ago

Check out:

Esri compact cache specification https://github.com/Esri/raster-tiles-compactcache

Esri tile cache specification https://github.com/Esri/tile-package-spec

That should give you all information you need to read a VTPK file. Projections are a bit of guesswork, but otherwise it is quite straight forward.

/Måns

On 3 Feb 2022, at 18:53, Thomas Halwax @.***> wrote:

 @MannusEtten The value {} for fields is hard-coded in the current version. I had a look at your example file and the file metadata.json (folder "p12") does not contain any values for fields either:

"vector_layers": [ { "id": "bouwtekening", "description": "bouwtekening", "minzoom": 0, "maxzoom": 21, "fields": {} } ] The examples provided by @spatialillusions TPK2GPKG do not even contain a metadata.json file.

Since I need to reverse engineer the VTPK specification it's pretty hard to find the correct way to extract the field information.

The debug output of the transform command shows

VectorTileLayer { version: 2, name: 'bouwtekening', extent: 536870912, length: 22, _pbf: { buf: Buffer(31277) [Uint8Array] [ 26, 169, 244, 1, 10, 12, 98, 111, 117, 119, 116, 101, 107, 101, 110, 105, 110, 103, 18, 64, 18, 2, 0, 0, 24, 2, 34, 56, 9, 232, 184, 254, 134, 2, 158, 195, 255, 169, 1, 10, 203, 2, 174, 2, 9, 220, 2, 153, 2, 10, 201, 2, 172, 2, 9, 228, 7, 168, 13, 10, 207, 2, 165, 2, 9, 232, 10, 169, 13, 10, 229, 1, 181, 1, 9, 180, 7, 208, 11, 10, 133, 2, 234, 2, 18, 186, 6, 18, 2, 0, 1, 24, 2, 34, 177, 6, 9, 170, 153, 254, ... 31177 more items ], pos: 31277, type: 0, length: 31277 }, _keys: [ '_symbol', '_minzoom' ], _values: [ 11, 15, 16, 10, 7, 6, 18, 0, 19, 8, 13, 14, 3, 5, 2, 1, 17, 4, 9, 12, 20, 21 ], _features: [ 19, 85, 914, 2846, 2878, 6831, 10641, 11043, 11199, 11236, 14232, 14481, 16180, 16728, 18014, 18192, 18275, 23691, 29310, 30642, 30845, 31114 ] } Here we can find _keys that seem to contain the value '_symbol' as your metadata.json does. But what about _minzoom? And how should I guess the type of the field? Maybe I can do that by looking at the _values field.

In order to make a solid decision I would need more complex examples.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.

ThomasHalwax commented 2 years ago

@spatialillusions I'm aware of these spec files but couldn't find a description of how the fields information can be extracted reliably. Some VTPK files seem to have a metadata.json file and some may not (like the ones you provided). If you do have access to ArcGIS maybe you can create some more examples so I can work it out? Thanks!