Leverage Hardware Occlusion Queries in Tileset Traversal

[nithinp7]

In Cesium Native, we currently cull tiles based on the frustum and fog (distance). This means we are often loading tiles that are occluded by other tiles, especially in close-up views (e.g., in between buildings in a city). Luckily Unreal avoids draw-calls for many of these tiles using hardware occlusion queries. We could take it one step further and take the result of these visibility queries and feed them back to Cesium Native's traversal logic.

Some thoughts (will make this more concise when I get a chance):

A naive approach to start would be just avoid further refining tiles that are reported to be occluded, regardless of the SSE. If the tile is already refined and has loaded descendants, we can consider all those descendants as culled. In this approach, nothing changes about how or what we render in GltfComponent. The occlusion results that Unreal generates anyways will simply be piped back to the Cesium Native traversal.

• Rendered tiles should wait for the occlusion result, which may take several frames, before deciding to refine.
• If we skip rendering too many LODs (to jump straight to the target SSE) we may miss chances of occlusion culling the entire subtree.
• In this basic approach, we can't consider an occluded tile as culled itself. E.g., if we culled the tile and kicked it from the in-memory cache we wouldn't know when the tile was later un-occluded, since Unreal will no longer be updating occlusion queries on the tile.
• As the camera view changes, the occlusion result of parent tile needs to be used to occlusion cull out already loaded children. However, currently there is no guarantee that parent tiles stay loaded when descendants are being rendered, if they were ever loaded at all (again they may have been skipped over in favor of descendants during the original loading).
• This approach may require parents within some tolerable SSE band (e.g., < 4x target SSE) to stay loaded just so they continue to update their occlusion results. Only when they are loaded and updating their occlusion result can we know whether or not to cull their descendants. (It may not even be possible to retrieve occlusion results for a parent that we have SetVisibility(false)).

Unfortunately, this approach is awkward and is nonoptimal when parents are skipped or kicked in favor of descendants (descendants can only be occlusion culled if a parent is rendered and receiving occlusion results). It would be easier if we could get occlusion results for tiles that aren't necessarily loaded. Luckily hardware occlusion queries only require a bounding volume for the occludee, which we have before the tile is loaded.

• Maybe we can find a way to just feed bounding volumes into the occlusion culling pipeline without needing to load the tile or create all the gltf render resources.
• Hopefully we can find a clean hook-in point to the existing occlusion culling pipeline. There is the regular hardware occlusion queries and there is hierarchical z-buffer occlusion culling, my understanding is that hzb is faster but more conservative.
• Less ideally, we can manually compute occlusion by queueing up compute shader passes to draw the bounding volume into a copy of the last frame's depth buffer (for hzb a "min" mip-mapped depth buffer). We can do this in hierarchical order to early-out once an entire subtree is known to be occluded.

I am going to experiment a bit on the Unreal side to get a better sense of how and if the occlusion system is exposed. If it isn't I will try to form an educated guess of how difficult it would be to set up a custom gpu-based occlusion system using compute shaders. In general it seems straight forward, but it depends on the amount of Unreal API-wrangling that is necessary.

Some relevant links:

• https://udn.unrealengine.com/s/question/0D52L00005EJHyrSAH/calculate-percent-of-rendered-object-that-is-visible-in-viewport
• https://docs.unrealengine.com/4.26/en-US/API/Runtime/RHI/RHICreateRenderQueryPool/
• https://github.com/EpicGames/UnrealEngine/blob/9e0f5f2a419a75b83392c44bf75462366ca8c1e2/Engine/Source/Runtime/Renderer/Private/SceneOcclusion.cpp#L377

[kring]

I was telling Patrick a little bit about this, and he's very interested as he spent some time thinking about this problem for his MS thesis. From Patrick:

It could also be worthwhile to use the occlusion to drive the LOD, i.e., if a tile is mostly occluded then use a lower resolution tile. Of course, it could impact visual quality, but we would have to test to see the tradeoff in practice and we could also make this an option and parameterize it.

I looked into this for my MS thesis, but I don't think I ever implemented. Search for "VMSSE" and see Section 3.4 - http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.371.4703&rep=rep1&type=pdf

It references this paper: https://www.researchgate.net/publication/220845198_Optimized_HLOD_Refinement_Driven_by_Hardware_Occlusion_Queries

I know the third author a bit if we ever wanted to talk to them; he and his students did a lot in HLOD and out-of-core rendering in the 2000's (and maybe later but I haven't tracked their work recently).

[kring]

More from Patrick:

BTW I had one more idea about occlusion culling...sorry...I am the worst...so the queries are pretty low overhead because they are async, just a few vertices, and with pass through fragment shaders...however, they do end up with a ton of just raw fillrate so are you guys turning them off at all when they don't work well, e.g., (1) above a certain altitude, (2) based on dot product of view vector and local surface normal, (3) based on how success they were over the past n frames??? Fun, fun, fun, I am so jealous!

[nithinp7]

Implemented in #848.