Simplify geometries at low zoom levels

[1ec5]

Boundaries add a lot of heft to the vector tiles, especially because each iteration of a boundary is represented as a separate polygon feature: https://github.com/OpenHistoricalMap/issues/issues/698#issuecomment-1954675311. #603 explores generalization across time, but before we embark on an innovative solution to that problem, we should simplify the geometry of each feature in the first place. The simplification should be more aggressive at lower zoom levels, with the highest zoom level containing unsimplified geometries.

The TOML file regurgitates the raw geometry and reuses the same geometry at every zoom level:

https://github.com/OpenHistoricalMap/ohm-deploy/blob/f0a1cfac4af4701859c1e4a739d4623b726b65bb/images/tiler-server/config/config.toml#L91-L131 https://github.com/OpenHistoricalMap/ohm-deploy/blob/f0a1cfac4af4701859c1e4a739d4623b726b65bb/images/tiler-server/config/config.toml#L324-L370

By comparison, OpenMapTiles sends the geometry through ST_Simplify with a resolution that scales by zoom level:

https://github.com/openmaptiles/openmaptiles/blob/b3c32321a39979ae8c0d73b548c41ecd7a0ac1a7/layers/boundary/boundary.sql#L71 https://github.com/openmaptiles/openmaptiles/blob/b3c32321a39979ae8c0d73b548c41ecd7a0ac1a7/layers/boundary/mapping.yaml#L8-L11 https://github.com/openmaptiles/openmaptiles-tools/blob/d241d4434811159bc82bcd2a94f11db5bbba89ae/openmaptiles/imposm.py#L5-L7

ST_Simplify uses the Ramer–Douglas–Peucker algorithm, which will result in some artifacts, because it only operates on individual geometries and doesn’t preserve topology. #701 is an alternative approach that would preserve boundary topology, at the expense of being able to reason about territories on the client side.

[danrademacher]

I think we should go ahead and pursue some simplification as described in this ticket and not wait on #701, and then pursue 701 separately. Seems like we should have some simplification at least through z10 and maybe beyond, regardless of what we might do in future with replacing polygons with lines.

Does that align with your thinking, @1ec5?

[1ec5]

This can proceed first, since it’s pretty straightforward. However, the topology issue will become apparent very quickly, because we’re styling the borders by outlining them instead of shading them in. If we simplify too aggressively, what you’ll see is a confusing mess of crisscrossing lines in places where boundaries aren’t perfectly straight and at right angles. The style might be able to compensate by faking a halo around the border (actually an opaque, faint, thick line), as OSM Americana does, but the real fix will be to draw boundary lines based on polyline geometries.

[danrademacher]

@Rub21 what do you think about introducing simplification in our boundary layers without also taking on #701?

This is something I would like to try on staging, but I'm not sure of the level of effort involved. Maybe just a simple addition to the Postgres queries and then we see what the results are? Would we need to flush and re-render all currently cached tiles to see any difference?

[Rub21]

This is something I would like to try on staging, but I'm not sure of the level of effort involved. Maybe just a simple addition to the Postgres queries and then we see what the results are? Would we need to flush and re-render all currently cached tiles to see any difference?

That make sense, I will take a look on it, we can add a function to postgis and run it in config.toml. i can take tis on my plate.

[Rub21]

I have applied simplification on staging apart from updating the Tegola version in the container. Here is the configuration: according to the zoom level 👇

    # Zoom levels 0-2,Tolerance=500 => ~1000 meters
    # Zoom levels 3-5,Tolerance=200=> ~500 meters
    # Zoom levels 6-7,Tolerance=100=> ~200 meters
    # Zoom levels 8-9,Tolerance=50=> ~100 meters
    # Zoom levels 10-12,Tolerance=20=> ~50 meters
    # Zoom levels 13-15,Tolerance=5=> ~20 meters
    # Zoom levels 16-20,Tolerance=1=> ~5 meters

Making the comparison, the configuration does not modify the geometry, and the tiles have been reduced in size. 👇

Productions:

Staging - new version of Tegola(0.19.0) an simplification the admin boundaries

here a list of comparison for some tiles: https://gist.github.com/Rub21/86b5f337cd8cbe6d1d42eecb283cc384

It appears that simplification can also be applied to areas like land use and water bodies without significantly affecting the geometry, But I am still testing to see if we can do more simplification on the boundaries to reduce the size of the tiles.

[1ec5]

It appears that simplification can also be applied to areas like land use and water bodies without significantly affecting the geometry,

By the way, this could have a significant impact on tiles that include coastlines.

[Rub21]

I did simplification on the coastlines, (land layer), and make a comparison between the current production and staging, the size reduction looks promising.

https://gistpreview.github.io/?bbbdd21561b392c9744658e4446308e3#6.04/50.455/-126.703

[Rub21]

For some small areas, there is an impact of simplification, maybe that's something that we need to discuss. https://gistpreview.github.io/?ece5e6ea67fc458f90365ddf9eb70b62#4.89/46.23/-61.62 , 👇

[danrademacher]

Admin lines look great.

Coast lines we are losing some small islands, so we want to preserve those,

Example: https://www.openhistoricalmap.org/#map=13/49.4012/-126.2418&layers=O&date=1981-06-23&daterange=1824-01-01,2024-12-31 vs https://staging.openhistoricalmap.org/#map=13/49.4013/-126.2418&layers=O&date=1981-06-23&daterange=1824-01-01,2024-12-31

[Rub21]

Admin boundaries and coastlines have been simplified successfully and deployed in production!

[Rub21]

Per chat with Dan and Sanjay, we are going to work on simplification on others features, such water body, water lines, landuse, etc.

[Rub21]

I did a simplification of water body, water lines, landuse, it can be compare with production at https://openhistoricalmap.github.io/tiler-compare/#2/0/0

[danrademacher]

Here's a good place where the differences are somewhat visible in the tiles: https://openhistoricalmap.github.io/tiler-compare/#7/59.355/11.068

A little hard since the data itself is different between prod and staging (that is, polygons in one might not be in the other).

I think the polygon on the left is more complex than the polygon on the right

But we have more data in staging (I assume in response to other tickets asking for more stuff on the map), so tiles on staging are actually larger than prod in this area:

I am actually having trouble finding areas where staging tiles are significantly smaller in size than production, but I would expect to see that difference if the simplifications are enough to improve performance. Staging tiles are also slow to render, though I assume that's because we have fewer resources on staging.

@Rub21 can you suggest any locations where you see the impact of this simplification with lower tile sizes?

[danrademacher]

This is already deployed. Let's just see if any issues come up in future,