Anttwo
commented
6 months ago Hi @SeanGuo063,
Actually, using the point with the maximum density value is not a good idea for extracting a surface, for several reasons. Please let me explain them below.
- First of all, we are looking for a point that crosses the isosurface only in small neighborhoods of Gaussian splats within depth maps rendered with the Gaussian Splatting rasterizer. However, these depth maps are not 100% accurate, so we cannot be sure that there is an isosurface-crossing point in the sampling neighborhood. For instance, there might be only points with very low density in the neighborhood, such that the point with maximum density would have a density value of 0.1, which is too low. We need to set a threshold to ignore points with low density.
- Secondly, we want some homogeneity among the extracted points to get a smoother surface. If we always choose the point with maximum density in the neighborhoods, we cannot be sure that they have similar density: one might have a density of 0.7, and another might have a density of 1.7. Therefore, selecting a predefined level helps to get a better-looking surface.
- Thirdly, and most importantly, the maximum value is a poor choice for an isosurface extraction. Let me illustrate this with a simple example of a single Gaussian in 2D:
If we select the surface with maximum density value, we would extract only a single point from this Gaussian blob, which would be the center point of the Gaussian. However, our brain does not perceive a single point when looking at a Gaussian; we see an ellipsoid, with a fuzzy border. Therefore, "the true surface" as our brain perceives it actually corresponds to a isosurface with a lambda value lower than 1! To select a good lambda value for the isosurface, we have to answer this question: Where should we draw the border that defines the ellipsoid when looking at this image? Intuitively, lambda=0.5 or lambda=0.3 are generally pretty good guesses as you can see in the picture. It turns out that experimental results confirm this intuition, as we show in the paper.
I have a question of mesh extraction. Why using lambda，rather than finding the point with the maximum density value along the ray, since max density should theoretically be corresponding to the true surface.