buaavrcg / LEGaussians

Pytorch Code for "LEGaussians: Language Embedded 3D Gaussians for Open-Vocabulary Scene Understanding"
https://buaavrcg.github.io/LEGaussians/
MIT License
100 stars 14 forks source link

a potential bug in the diff-gaussian-rasterization #10

Closed LiuJF1226 closed 2 months ago

LiuJF1226 commented 2 months ago

Thank you for the interesting work. I notice that NUM_CHANNELS is modified to 12. But the bg_color is still [0,0,0] of 3 channels. Therefore, this line can lead to array index out of bounds for bg_color.

Chuan-10 commented 2 months ago

Thank you for pointing that out! I'll fix it.

LiuJF1226 commented 2 months ago

Setting bg_color to [0,0,0,0,0,0,0,0,0,0,0,0] here may be helpful. But this raises another problem. Directly setting the background for features and uncertainty to zero, or binding a learnable background for features and uncertainty, which one could be better?

Chuan-10 commented 2 months ago

It may depend on the type of scene.

For bounded and simple scenes, such as synthetic scenes with a dark background, setting the background to all zeros might be helpful since the decoder doesn’t need to fit a complex distribution.

For unbounded and large scenes, such as urban environments where the background has specific meaning, having a learnable background for both features and uncertainty might be more effective in fitting the decoder’s distribution.

It is just my intuition, if there is any mistake or problem, please point it out and discuss it more.

LiuJF1226 commented 2 months ago

Thank you for your reply. For generalization ability, I think it is more reasonable to employ a learnable background for both features and uncertainty. But It needs more experiments for validation.