Geometry that should be clipped sometimes appears at certain camera angles/distances

kring commented 1 week ago

Reported multiple times on the community forum:

kring commented 1 week ago

I can reproduce this problem, and I have a working theory about what is happening.

First of all, tiles that are entirely inside the clipping polygon seem to no longer get excluded. This was probably broken in v2.4.1, which means it was probably broken by a cesium-native change in this set of changes: https://github.com/CesiumGS/cesium-native/compare/c9cf5430d5a1d5a2bbdf667169e2c47316f7a8b0...e868c5c85b2de61ded38ff21a88e49f3c6617a95

But that doesn't explain why the phantom geometry appears, because the clipping material layer should discard it regardless.

The immediate reason that that geometry appears is because when we sample the polygon raster overlay, we once-in-awhile erroneously see an alpha 0.0 (or less than the mask cutoff value of 0.5, anyway). This is very surprising because this texture doesn't even have an alpha channel! So why in the world would we sometimes see an alpha of 1.0 (as we should), and sometimes see other values. It doesn't make a lot of sense.

My working theory is that this only happens in a tile that is entirely inside the clipping polygon (that's the only time I've seen it happen, at least). And in that scenario, we create a 1x1 texture for the clipping mask. That seems like it should be fine, but we have previously seen cases where Unreal materials just straight up don't work with 1x1 textures. Switching to 2x2 textures has fixed those previous problems. I wonder if it will help here, too.

kring commented 1 week ago

Switching to 2x2 or even 4x4 textures didn't help.

The actual problem is that the polygon raster overlay isn't being mapped to these problematic tiles at all. This is strange because a polygon raster overlay covers the entire globe. The immediate problem is that getTileBoundingRegionForUpsampling in TilesetContentManager.cpp is returning a region with a north coordinate that is less than its south coordinate. This commit fixes it, and I believe it's a good change - I did the same thing previously for the Clips feature in ion. But I can't entirely explain why it fixes it, so I don't trust it as a complete fix yet.

First of all, tiles that are entirely inside the clipping polygon seem to no longer get excluded.

The reason for this is described in CesiumGS/cesium-native#914.

j9liu commented 1 week ago

kring commented 1 week ago

I think I finally understand what's going on here. In Google Photorealistic 3D Tiles, we often have tiles that look like this:

Where the border shows the tile boundary and the blue blobs are some geometry within it. So there's a lot of blank space, but also some geometry. The bounding volume is tight-fitting. For example, this tile might be at elevation and the geometry is treetops.

Now in order to do polygon clipping, we have to map a raster overlay to this tile. We assign UV (0,0) to the bottom left, and UV (1,1) to the top right, and UV coordinates for vertices in between are calculated based on the position of the vertex in the raster overlay's projection. So far, so good.

Now let's say this is a leaf tile. If we want to hang a more detailed raster overlay on it, we need to "upsample" it, which means divide its geometry into four new, smaller tiles. It looks like this:

Notice that the new tile in the lower left only has geometry on the right side. The box around it is not tight-fitting. Now we need to generate raster overlay texture coordinates for this new tile. As an optimization, we don't go compute the bounding rectangle for it first. Instead, we assume that when the tile has a bounding volume which is a region, we can trust it. It may not be optimal, but it should be good enough. We generate the texture coordinates based on the full box, even though the geometry only occupies part of it. We do this in order to avoid making two passes over the geometry: one to compute the rectangle, and a second to compute the texture coordinates using the rectangle.

The texture coordinate generation process has two outputs besides the texture coordinates themselves: the rectangle that was used to compute the texture coordinates (which may not be tight fitting and that's ok), and the actual tight-fitting bounding rectangle. The tight-fitting bounding rectangle is computed nearly for free while computing the texture coordinates (but by then it's too late to use this tight-fitting rectangle for the coordinates themselves without another pass).

So far, there's a bit of wasted UV and texture space in this tile, but nothing catastrophic has happened. But what happens when we need to upsample that tile for further raster overlay detail?

Here's where we run into problems. We have to decide on the bounding regions for the four upsampled children. And currently, it does this: it divides the parent's tight-fitting bounding rectangle at the point where the texture coordinates would be exactly (0.5,0.5). Remember: the texture coordinates are based on the non-tight-fitting volume. So it looks like this:

See the problem? The center split point is not inside the bounding rectangle at all. A bounding rectangle for the southwest tile that is created like this ends up being total rubbish:

CesiumGeospatial::GlobeRectangle(
  parentRectangle.getWest(), // west
  parentRectangle.getSouth(), // south
  center.longitude, // east
  center.latitude) // north

The bounding rectangle doesn't properly have west less than east and south less than north. Depending on some details, it might be considered empty, or wrapping the wrong way around the entire Earth past the anti-meridian.

kring commented 1 week ago

What do we do about this? Well, there are basically two choices:

Choose a center point for subdivision that is at the center of the tight-fitting bounding volume instead of in the center of the previously-computed texture coordinates, or
Continue to subdivide at the center point of the texture coordinates, but use the rectangle that was used to compute the texture coordinates instead of the tight-fitting rectangle.
Bonus third option! Do two passes over the geometry in order to determine the tight-fitting bounding box before computing texture coordinates, so there is never a disagreement between them.

I don't like (3) very much. It's expensive (every vertex has to be transformed to ECEF and converted to longitude/latitude/height) and usually unnecessary.

(2) is much easier than (1), because the upsampling algorithm currently always assumes it is splitting at (0.5,0.5) (though this wouldn't be too hard to change). This is what I did previously for Cesium Clips, and it is what the commit I mentioned above does (the one that fixed the problem but I couldn't explain why at the time).

Is it the best solution, though? Well, it'll lead to three out of the four upsampled tiles being blank in the example shown above. That's not a huge problem, though.

I think (2) will have better LOD behavior than (1). With (1), we end up with a very small tile with a large-ish geometric error - half the LOD of its parent. Which means that if you zoom in close to that tile so that it fills the screen, and the tile is further upsampled, you can end up with a giant grid of tiles on the screen. This might point to shortcomings in our geometric error selection for upsampled tiles, but it's not obvious what would be better. Anyway, it's not great, and (2) doesn't have that problem.

So that's a point against (1). I can't think of much downside to (2). So (2) it is!

kring commented 1 week ago

This is what I did previously for Cesium Clips, and it is what the commit I https://github.com/CesiumGS/cesium-unreal/issues/1461#issuecomment-2188354252 does (the one that fixed the problem but I couldn't explain why at the time).

Oops the fix in Clips was actually closer to (1), and kind of broken because it cuts the geometry at a different place from where it divides the bounding volume.

CesiumGS / cesium-unreal

Geometry that should be clipped sometimes appears at certain camera angles/distances #1461