autonomousvision / mip-splatting

[CVPR'24 Best Student Paper] Mip-Splatting: Alias-free 3D Gaussian Splatting
https://niujinshuchong.github.io/mip-splatting/
Other
1.01k stars 65 forks source link

About Eq.9 #19

Closed chensiyuan030105 closed 4 months ago

chensiyuan030105 commented 7 months ago

In Eq.9, it is the square of covariance plus the square of 3D Filter, but in the code, it is the square of exp(scaling) plus the square of 3D Filter squared. Why?

3DGS-MIP-2 3DGS-MIP-1
niujinshuchong commented 7 months ago

Hi, we first get the original scale of the Gaussian by self.get_scaling and after applying square it becomes the variances (left term in L105). Then we apply the 3D filter (sum in L105 in your screenshot, eq.9 in the paper). Finally applying sqrt to the variance results in scales after 3D filtering.

chensiyuan030105 commented 7 months ago

Thank you for your reply! I have some other questions.

  1. In the article, you proposed three drawbacks of 3DGS : high frequency artifacts, dilation, aliasing. And the article proposed 3D Smooth filter and 2D Mip filter to solve it. After reading the article, I think 3D Smooth filter solve high frequency artifacts by limiting the smallest scale of gaussian (zooming in) and 2D Mip filter solve dilation and aliasing (zooming out). However, in games 310's representation, I get the information that 3D Smooth filter solve both high frequency artifacts(zooming in) and dilation(zooming out), and 2D Mip filter solve aliasing(zooming out). It confused me a lot.
chensiyuan030105 commented 7 months ago
image
chensiyuan030105 commented 7 months ago
image
chensiyuan030105 commented 7 months ago

From the ablation study, I think it is 2D Mip filter solves dilation, not 3D Smooth filter, which is different from the conclusion from the games 310

image
chensiyuan030105 commented 7 months ago
image image

And how about dilation(zooming out)?

Yang-Xijie commented 6 months ago

I think 3D Smooth filter solve high frequency artifacts by limiting the smallest scale of gaussian (zooming in) and 2D Mip filter solve dilation and aliasing (zooming out).

From my own point of view,

chensiyuan030105 commented 6 months ago

I agree with your first and second statement. However, for dilation, I think the reason for the artifacts caused by dilation is too small gaussian(too high frequency) for zooming out.

chensiyuan030105 commented 6 months ago

The coefficient of gaussian filter in EWA splatting is a good correction but not contribute for solving dilation.(I think)