mapbox / mapbox-gl-js

Interactive, thoroughly customizable maps in the browser, powered by vector tiles and WebGL
https://docs.mapbox.com/mapbox-gl-js/
Other
11.19k stars 2.22k forks source link

Add Ambient Occlusion layer to enhance 3d scene depth perception #10254

Open karimnaaji opened 3 years ago

karimnaaji commented 3 years ago

Prototype

To enhance 3d scene perception (similarly to fog proposed in https://github.com/mapbox/mapbox-gl-js/issues/10205) we could introduce the use of an Ambient Occlusion (AO) layer, also called Sky View Factor. It's an indirect lighting term representing how much a point in space sees the sky for a surface on the ground and can be complementary to direct lighting (hillshading). It is called indirect lighting as it represents how much light bounces from other locations and increases the perceived brightness of a surface, compared to more direct or local lighting only shading with information gathered from the currently shaded point. Since ambient occlusion has a more global effect, areas that are harder to reach from the skylight will become darker compared to areas that are very exposed to it.

Screen Shot 2021-01-04 at 3 24 33 PM

AO better retains depth cues and increases overall depth perception of a scene. Since this term varies based on how occluded a shaded point is, it allows for an understanding of how steep, narrow, and open terrain areas are and better layers terrain features between each other. The main effect is pushing some areas in the background while retaining others in the foreground. This improves landscape understanding which isn't easily representable with typical Hillshading, as it suffers from flattening everything at the same level of depth.

We can use a line-sweep method from this paper to sweep the DEM in a few directions, and for each of the sweeps build a list of horizon points and applies an occlusion factor which is then averaged for all sweep directions. Here are some results from prototyping:

Screen Shot 2020-10-22 at 3 04 12 PM

Screen Shot 2020-10-23 at 9 28 10 AM

Screen Shot 2020-10-21 at 6 26 56 PM

Screen Shot 2020-10-21 at 6 25 21 PM

To create an AO term we've explored two different techniques: standard deviation with local hypsometric tints and the line sweeps which gave the best results in the prototype. I also have the intuition that the line sweeps could be later used to add shadows, as a single sweep with a reduced falloff distance could approximate shadows, as seen here:

Screen Shot 2020-10-19 at 11 54 27 AM

Design considerations

Risk

Since this lighting term is global it requires tile neighboring information. We can have large backfill lookup on neighboring tiles (100px gave good results as proposed by @arindam1993 ). Another unexplored option may be to build and store the 2d elevation convex hull result at tile boundaries. Though, the term is constant enough that it could justify server side implementation.

rreusser commented 3 years ago

Regarding whether to compute on the client vs. server:

Advantages of computing on the client include:

Advantages of computing on the server:

I don't know the answer, but as much as I like client-side computation, I was certainly interested to learn about the alternatives.

/cc @dnomadb

astojilj commented 3 years ago

Client side seems preferable - would be nice to include fill-extrusions and custom layer models.