slothfulxtx / diff-gaussian-rasterization

Differentiable gaussian rasterization with depth, alpha, normal map and extra per-Gaussian attributes, also support camera pose gradient
Other
193 stars 12 forks source link

Rasterization Question - Meaning of radii #7

Closed robofar closed 6 months ago

robofar commented 7 months ago

Hello. I am confused about the meaning of variable radii from this code:

from diff_gauss import GaussianRasterizationSettings, GaussianRasterizer

rendered_image, rendered_depth, rendered_alpha, radii = rasterizer(
    means3D = means3D,
    means2D = means2D,
    shs = shs,
    colors_precomp = colors_precomp,
    opacities = opacity,
    scales = scales,
    rotations = rotations,
    cov3D_precomp = cov3D_precomp
)

Could you please explain me what is the meaning of radii for each Gaussian and what its value represent in context of one Gaussian? I see that output of this variable is:

tensor([15,  7, 15,  ...,  4,  5,  3], device='cuda:0', dtype=torch.int32)

So for each Gaussian in the scene, there will be one radii value. What does this value represent?

P.S. It seems that radii is plural of radius, which I didn't know :D.

But still I am confused about what value represent? Radius of Gaussian in which unit and in which space (2D/3D)? What is this value if Gaussian is non-isotropic? Maybe none of this is true, so I am confused about this variable meaning.

slothfulxtx commented 6 months ago

Hi, as i understand, the radii can be considered as "the bounding box" of each 2D Gaussian projected from 3D Gaussian using EWA splatting. It determines which 16*16 tiles may intersect with the 3D Gaussian. The following lines describe how we compute the radii

  float mid = 0.5f * (cov.x + cov.z);
  float lambda1 = mid + sqrt(max(0.1f, mid * mid - det));
  float lambda2 = mid - sqrt(max(0.1f, mid * mid - det));
  float my_radius = ceil(3.f * sqrt(max(lambda1, lambda2)));

After we obtain the "bounding circle", we create a copy of 2D Gaussian for each tiles that intersect with the circle and then sort 2D Gaussians according to their depths

duplicateWithKeys << <(P + 255) / 256, 256 >> > (
    P,
    geomState.means2D,
    geomState.depths,
    geomState.point_offsets,
    binningState.point_list_keys_unsorted,
    binningState.point_list_unsorted,
    radii,
    tile_grid)
......
CHECK_CUDA(cub::DeviceRadixSort::SortPairs(
    binningState.list_sorting_space,
    binningState.sorting_size,
    binningState.point_list_keys_unsorted, binningState.point_list_keys,
    binningState.point_list_unsorted, binningState.point_list,
    num_rendered, 0, 32 + bit), debug)