vector-tile-js v3 proposal

[mourner]

This is a proposal for a breaking overhaul of vector-tile-js that will simplify the API, significantly improve performance, and streamline interoperability between related libraries. Previously: https://github.com/mapbox/mapbox-gl-js/issues/10501

Vector Tile JS spec

There will be a specified format for representing vector tiles as pure JSON. Both the output of vector-tile-js and all related libraries (e.g. geojson-vt, supercluster) and the input for encoding (currently done by vt-pbf) will follow this format, now shared across the ecosystem.

This will ensure consistency and easy interoperability between vector-tile-js, geojson-vt, supercluster and mapbox-gl-js for both reading and writing vector data and avoid redundant wrappers and conversions between formats, improving performance and memory footprint.

JSON objects will replace the current hierarchy of classes (VectorTile, VectorTileLayer, VectorTileFeature). In particular, feature.loadGeometry() method will be replaced with feature.geometry property, making potential lazy decoding of features an internal implementation detail (e.g. using JS getters).

Flat geometry encoding

Instead of a special Point class to represent points in features, vector-tile-js and related libraries will use flat arrays of numbers ([x, y, x, y, ...]). This will minimize memory footprint when dealing with vector data, avoiding excessive array allocations in JS.

This is the most disruptive change, requiring an overhaul of all downstream code that processes vector tile geometry (specifically, bucket code in mapbox-gl-js), but with a potentially huge performance benefit.

Consistent geometry nesting

Instead of representing any type of geometry as an array of arrays of points, Vector Tile JS will use different nesting depending on type:

• Array<number> for points
• Array<Array<number>> for lines
• Array<Array<Array<number>>> for polygons

This will allow us to include polygon nesting information in the output, avoiding ring classification in donwstream code to simplify polygon processing and reduce redundant conversions. It will also reduce redundant array wrapping for point features.

We could also go futher and collapse the outer wrapping array for single-line and single-polygon features, further improving memory footprint at the expense of more involved handling code downstream, but first we need to test whether this overhead is significant in practice.

Vector Tile encoding

After we unify the API for representing vector tiles in JS, vt-pbf will no longer feel right as a separate library — we can merge it’s functionality into this one, which will do both encoding and decoding.

Other breaking changes

• feature.toGeoJSON(x, y, z) will likely become toGeoJSON(feature, x, y, z) — this will make it usable for other producers of vector tile data like geojson-vt and supercluster, and fit the new paradigm of features being accessed as JSON objects.
• Should we remove feature.extent? It’s a layer property which is currently also duplicated for every feature for unclear reasons.
• Should we remove feature.bbox()? It doesn’t appear to be used anywhere downstream, and there’s not much overhead to iterating over geometry to get a bbox instead.

cc @mapbox/gl-js @kkaefer

[mourner]

Types for what the VT JSON spec could look like:

type FeatureBase = {id?: number, properties: {[string]: string | number | boolean}};

type PointFeature   = FeatureBase & {type: 1, geometry: Array<number>};
type LineFeature    = FeatureBase & {type: 2, geometry: Array<Array<number>>};
type PolygonFeature = FeatureBase & {type: 3, geometry: Array<Array<Array<number>>>};

type Feature = PointFeature | LineFeature | PolygonFeature;

type Layer = {extent: number, features: Array<Feature>};

type Tile = {layers: {[string]: Layer}};