graphdeco-inria / gaussian-splatting

Original reference implementation of "3D Gaussian Splatting for Real-Time Radiance Field Rendering"
https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/
Other
13.95k stars 1.8k forks source link

Question About Discontinuous Color #770

Open xerokjames opened 5 months ago

xerokjames commented 5 months ago

Hello, thanks to your great work! I have a question about the continuity of rendering color. When I reconstructed a model of a reflective object and rendered it according to my camera path, I found that the colors in some regions were not continuous. In other words, when a camera position made a small change, the colors in these areas had a large jump. I analyze that this is most likely because of the sorting mechanism. I first found that the transparency of some 3d gaussian points in the reflective regions is close to 0.5, neither high nor low. Sometimes when I change the camera position in a small value, the order of these gaussian points changes, which leads to a jump in the cumulative transmittance, which leads to a jump in color. The sorting process itself is discontinuous, but is there a continuous way that can achieve the similar effect as sorting? Looking forward to your answers very much!

PanagiotisP commented 5 months ago

That's a known limitation of the current algorithm; sorting happens once, based on the primitives' centers, not taking into account the extent, which can lead to these popping artifacts. The "continuous way" (that is, truly multi-view consistent) would be to have a per-pixel sorting step, but that's too costly. There is this work that tackles this problem exactly, by applying some kind of resorting in a hierarchical manner.