RFC: data-driven styling for fill-extrusion-opacity and robust opacity blending

[arindam1993]

We currently do not support data-driven styling for fill-extrusion-opacity, and transparent fill-extrusions that overlap each other do not blend correctly, with the extrusion closer to the camera culling the one behind.

Both of these issues stem from the fact that we do not render fill extrusions in an order defined by their distance to the camera. This RFC presents a few options for implementing this, comments and thoughts would really be appreciated!

Part 1) Handing tiles:

Sort visible tiles every frame based on distance to camera:

(calling this z-sorting from here on out, slightly confusing since we overload the term z with zoom level )

Since every tile is a separate draw call we can sort on the CPU before entering the draw loop for each tile. My intuition is that we should have enough CPU budget for this assuming roughly ~20-50 visible tiles on the screen, and we'd only do this when fill extrusions need to be rendered, typically at higher zoom levels, and we might even win some performance back by rendering front to back for opaque extrusions.

Part 2) Handling extrusions within the tile:

z-sort each feature on the CPU:

Doing this every frame will definitely be too expensive, but I was thinking we could precompute multiple sort-orders for for a a few different values of azimuth ( which would be in the range [0, 180] degrees) on the worker when we build the bucket. Then every frame, we can perform an O(1) lookup for the sort-order by taking Math.round(camera_azimuth/num_sort_orders). Illustration, each black arrow represents a pre-computed sort order for that azimuth:

Pros:

• No special hardware support required
• Easier and simpler to implement(?)
• Less z-fighting coz we render them one at a time

  Cons:

• Additional drawcall overhead, each extrusion would have to drawn with a separate draw call.
• Slower load times for the extrusion layer.
• Potentially inaccurate sorting since we're approximating based on closest available azimuth.

- OR -

Weighted, blended order-independent transparency:

http://casual-effects.blogspot.com/2015/03/implemented-weighted-blended-order.html

Multi-pass technique, that just needs a separate pass for opaque and transparent objects. But this requires OES_texture_half_float support :(, WebGL stats says only 69% of mobile devices support it.https://webglstats.com/webgl/extension/OES_texture_half_float

style-spec dependencies:

if we decide to do this, it shouldn't be the default behavior because it will change how existing maps look and maybe make them much noisier. We should probably have a new style-spec property that explicitly enables this. Most 3D engines, define separate transparent: boolean and opacity: float parameters. This lets the designer have control by by allowing rendering of translucent extrusions that actually obscure each other ( our current implementation ).

[astojilj]

Both of these issues stem from the fact that we do not render fill extrusions in an order defined by their distance to the camera.

With current approach, we render only surfaces/faces closest to the camera (using depth buffer). Are you reconsidering about enabling both culling back faces and rendering front and back faces?

Otherwise, sort visible tiles every frame based on distance to camera looks like a favorable approach (performance).

[mpulkki-mapbox]

This is very interesting topic and a difficult problem to solve properly! :)

We're currently doing a depth pre-pass for translucent materials. I don't know the full motivation behind the approach but it fixes the self occlusion of non-convex meshes and it allows us to render translucent meshes without sorting them first. Doing a proper depth-sorted rendering with a self-occlusion constraint is challenging.

if we decide to do this, it shouldn't be the default behavior because it will change how existing maps look and maybe make them much noisier.

Removing the occlusion culling of buildings does indeed increase the noise. Here's an example (without self-occlusion):

[arindam1993]

With current approach, we render only surfaces/faces closest to the camera (using depth buffer). Are you reconsidering about enabling both culling back faces and rendering front and back faces?

I think we should definitely cull back faces, the tricky part as @mpulkki-mapbox mentioned is going to be addressing self-occlusion of non-convex front faces, if we want to do it. We could do that by applying our current 2-pass technique but for each extrusion. but that will probably be too slow. But its something we can let the designer choose for a specific layer, or for specific features by allowing a transparent: boolean paint property.

Otherwise, sort visible tiles every frame based on distance to camera looks like a favorable approach (performance).

Do you have any thoughts about handling order within the tile?Part 2 in the first post.

This is very interesting topic and a difficult problem to solve properly! :)

Yes :( , I don't think we should aim for a full scale highly performant order independent transparency just-yet but the main motivation for something like this would be to eliminate weird gotcha's like https://github.com/mapbox/mapbox-gl-js/issues/9046 https://github.com/mapbox/mapbox-gl-js/issues/8688 https://github.com/mapbox/mapbox-gl-js/issues/7785

[ahk]

I have had issues with transparency and occlusion on every 3D project I've worked on and it's always been a balance among serious tradeoffs. I'm excited for people to be exploring this topic! I've learned some good stuff already from this thread.

(Part 1) Makes sense and I'd be excited to try this out and for example see if the overdraw gains outweigh the CPU cost. Intuitively I think they would, although we are quite often CPU bound right?

(Part 2) I think this is interesting but I have a bit of a hard time visualizing what it would be like during interaction. Is it right to think as the Camera rotates around a target point we'd see popping/reordering as we pass azimuth thresholds?

I've done something like Weighted, blended order-independent transparency before and it worked alright, but I think the device support precludes it. Why do we need OES_texture_half_float for it?

(On noisy visuals) It looks pretty noisy to me in the example @mpulkki-mapbox posted. I think there may be a lot of design value in depth based occlusion. The idea of making per-feature settings to control opacity makes it more interesting potentially. It would be nice to have a prototype up where we could compare the two during interaction. Still frames don't tell the full story right?

(On draw call frequency) In general I think we have room for more draw calls. I saw that based on my experience with adding sort-key implementations. it doesn't seem like an issue that precludes trying it out.

[karimnaaji]

Nice recap @arindam1993, it's definitely a challenging topic.

Sort visible tiles every frame based on distance to camera

+1 on that for opaque related features and potential early-z rejection, perf for that could be tested if you easily allow to disable sorting in a debug toggle, and would probably be more critical on mobile.

Multi-pass technique, that just needs a separate pass for opaque and transparent objects. But this requires OES_texture_half_float support :(, WebGL stats says only 69% of mobile devices support it

I think that McGuire's technique does all the final compositing offscreen? So another potential hardware requirement would be the use of multisample render buffer storage (renderbufferStorageMultisample I believe) to not loose the anti-aliasing that we get by default or aim towards adding our own AA pass altogether, which is fairly typical when starting to use offscreen rendering for such techniques.

[arindam1993]

I think that McGuire's technique does all the final compositing offscreen? So another potential hardware requirement would be the use of multisample render buffer storage (renderbufferStorageMultisample I believe) to not loose the anti-aliasing that we get by default or aim towards adding our own AA pass altogether, which is fairly typical when starting to use offscreen rendering for such techniques.

+1 I think we can do our own taa to compensate?

I made a quick prototype of OIT fro fill extrusions using McGuires weighting function. I doesn't do a separate opaque pass, or front to back rendering of any sort. Just a 2 pass OIT for the entire screen. https://jsfiddle.net/pkszocgj/4/ You can click on the buildings to change their color.

Even after fixing the overflow bugs the output stills looks very noisy, given have a lot of extrusions nested within one another.

However it looks fine at lower zoom levels, and seems useful in a navigation usecase to be able to highlight buildings that are occluded by other buildings.

[designevokes]

Agree with the higher zooms looking good (like gif above with blue building). For a highlighted building, it's crucial not to clip any colored faces of the extrusion even if buildings are stacked to those sides.

We have been playing with expressions to see if we could only render buildings on your street you are driving through, and nothing else. This keeps the noise dialed back also at z12-z17 zoom level for example.

Could there be a parameter that one could pass with run-time styling to decide when to allow for transparency in the building extrusions. That way default maybe is transparency off for normal customers, and those desiring this could flag it to be on. Then it could be a Studio expression as well. Example: Only apply transparency to buildings at z18-22

[vlasovigora]

Hello @arindam1993 ! Great work. Could you please share the code of your prototype?