WorldHistoricalGazetteer / whg3

Version 3 beta
BSD 3-Clause "New" or "Revised" License
4 stars 4 forks source link

WHG Tileserver #80

Closed docuracy closed 5 months ago

docuracy commented 8 months ago

Purposes:

  1. Remove dependency on MapTiler & MapBox
  2. Mapping large datasets

The tiles can be served using the open-source TileServer GL if installed on a WHG server: the standard option includes server-side rendering of vector tiles into raster formats, but I think the light option would be more efficient and considerably less hungry of resources.

SEE ALSO: https://github.com/WorldHistoricalGazetteer/whg3/issues/94

Future Development:

Notes

docuracy commented 8 months ago

TODO Configure Nginx reverse proxy

server {
    listen 80;
    server_name tiles.whgazetteer.org;

    location / {
        proxy_pass http://localhost:8080;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}
docuracy commented 8 months ago

Cannot rely on feature-state for vector tiles. Tiled features need to be restyled with each tiled data-load, something like this:

const map = new maplibregl.Map({
  container: 'map', // container id
  style: 'your/style.json', // your style file
  center: [longitude, latitude], // starting position
  zoom: 12 // starting zoom
});

// Assume you have a 'your-layer-id' in your style with a property called 'featureId'
map.on('data', 'your-vector-tile-source-id', () => {
  // This event is triggered when a vector tile source loads or updates
  // You can dynamically style the features here based on your logic

  // For example, change the color of features with a specific ID
  map.setPaintProperty('your-layer-id', 'fill-color', [
    'match',
    ['get', 'featureId'],
    'desired-feature-id',
    'red',
    'default-color'
  ]);
});

// You can also use 'dataloading' event if you want to capture loading before rendering
map.on('dataloading', event => {
  // This event is triggered when a vector tile is being loaded
  // You can apply dynamic styling or modifications here before rendering
  // Access the loaded tiles using `event.tile`
});
docuracy commented 8 months ago

Examples of pages rendering Datasets or Collections, for implementing/testing vector tiled datasets:

Both of these rely at present on a FeatureCollection delivered by either /datasets/{id}/mapdata/ or /collections/{coll}/mapdata/, which includes everything required to build both the map and the table. These are problematic for large datasets/collections because (a) the geometries often account for a large proportion of the data bandwidth, and (b) parsing them for rendering on the map can be time-consuming. To get around this, I have upgraded the URLs noted above so that they can additionally handle a querystring of either ?variant=tileset or ?variant=nullGeometry. In the first instance, the FeatureCollection is pared down to the bare minimum required for tileboss to prepare an .mbtiles tileset. In the second instance, it is pared down to the bare minimum required for constructing the table in the browser. When a feature row is clicked in the table, the map will be repositioned based on the geometry returned by the getPlace API.

The id root property of each feature, already required for styling features using the Maplibre featureState functionality, is to be repurposed for vector tilesets to provide the linkage between features clicked on the map and corresponding table features. The URLs take care of adding those ids to maintain consistency between the tileset and the table data. [NOTE: the pid is now also being preserved in the tileset.]

Plan of Action

Upgrade tiler.js:

Generate tilesets:

# Dataset example
curl -X POST -H "Content-Type: application/json" -d '{"geoJSONUrl": "https://dev.whgazetteer.org/datasets/10/mapdata/?variant=tileset"}' http://localhost:3000/tiler
# Collection example
curl -X POST -H "Content-Type: application/json" -d '{"geoJSONUrl": "https://dev.whgazetteer.org/collections/11/mapdata/?variant=tileset"}' http://localhost:3000/tiler

mapAndTable.js makes the following API call:

$.get(`/${ ds.ds_type || 'datasets' }/${ ds.id }/mapdata`, function (data) { // ds_type may alternatively be 'collections'
            console.log('data', data)
            for (const prop in data) {
                if (!ds.hasOwnProperty(prop)) {
                    ds[prop] = data[prop];
                }
            }
            console.log(`Dataset "${ds.title}" loaded.`);
            resolve();
        });
docuracy commented 8 months ago

Overhaul Map Layers

The codebase to date has been using feature-state to add highlighting to selected features, but this technique cannot be applied to features in vector tile layers. Rather than develop a separate system for styling such features, which is fraught with complications, it makes sense to adopt a highlighting system that can be applied regardless of layer source type.

The simplest option seems to be to extend the whg_maplibre Map class (defined in whg_maplibre.js). A modified version of the layers currently defined in mapLayerStyles.js could be added by a new method, and a another new method added for highlighting a given array of pids, something like:

maplibregl.Map.prototype.updateLayerColors = function (layerId, pidArray) {
    var paint = this.getPaintProperty(layerId, 'fill-color');
    paint[2] = pidArray;
    this.setPaintProperty(layerId, 'fill-color', paint);
};

The layers would be defined as variants of this, and generated dynamically:

{
        'id': 'layer-id',
        'type': 'fill',
        'source': 'places',
        'source-layer': 'source-layer',
        'paint': {
            'fill-color': [
                'match',
                ['get', 'pid'], 
                [], '#ff0000', 
                '#808080'  
            ],
            'fill-opacity': 0.7
        }
    }

The method would have a clearHighlighting option.

Another method would ideally be developed to replace the temporal filtering function which is currently implemented.