Preserve precision of raster-dem source when rendering hillshade layers

[halset]

When zooming far in, it looks like both the mapbox.terrain-rgb and my own bathymetry based terrain rgb tiles have only meter vertical resolution. However, at least my bathymetry data set has higher resolution.

Here is an example with the bathymetry tiles.

Here is an example with mapbox.terrain-rgb.

Do you have an example where mapbox.terrain-rgb shows shadows from vertical sub-meter resolution?

[mollymerp]

hi @halset – the terrain-rgb encoding is precise to 0.1 m but the underlying data vary in their resolution. what zoom level are these screenshots from – I see this kind of jagginess and pixelation with overzoomed tiles (the maxzoom of the terrain-rgb tileset is 15).

[halset]

When looking at the raw dem tile for 14/8431/4555, there seem to be smooth transitions, but when looking at the rendered map in mapbox-gl-js (and -native), the rendering seem to be rounded to whole meters. This screen shot is from terrain-rgb 14/8431/4555 rendered as hillshade.

[tomass]

@mollymerp is zoom 15 built-in limit in mapbox-gl-js? I'm also getting visually bad results with higher zooms (with terrain-rgb tiles created by rio rgbify with --interval 0.01 from a very detailed source GeoTIFF).

At zoom 15.42 everything is fine:

If I zoom a little bit to 15.6 I'm starting to get artificial visual lines:

And if I zoom to 18, I get this:

As initial raw data is very precise, it would be nice to have hillshading up to level 18.

[mollymerp]

@tomass we removed the hardcoded maxzoom in #6055, but I confirmed we are losing sub-meter precision in how we're currently unpacking and storing the raster-dem tile's pixel data. I will play around with refactoring this to preserve the resolution of the source and see if we can maintain performance.

[tomass]

Thank you @mollymerp In the meantime to those possibly looking for a temporary solution to this: decreasing maxzoom value in terrain rgb mbtiles json file to 15 or even 14 makes hillshade being scaled from zoom 15 to 18 which makes it lose sharpness, but it looks much better:

[astojilj]

@andrewharvey ,

Thanks for verifying that #8694 partially fixes the issues. Given that #8694 passes the original values, we'll need to check if the shader computation (on original data) introduces precision loss, I could use iOS Metal GPU shader debugger once we port #8694 to mapbox-gl-native.