buaavrcg / LEGaussians

Pytorch Code for "LEGaussians: Language Embedded 3D Gaussians for Open-Vocabulary Scene Understanding"
https://buaavrcg.github.io/LEGaussians/
MIT License
100 stars 14 forks source link

Issue with uncertainty variable #15

Open smileyenot983 opened 1 month ago

smileyenot983 commented 1 month ago

Hello!

I have observed following line in your code https://github.com/buaavrcg/LEGaussians/blob/main/train.py#L139

I have used the modification of this code and this line is failing sometimes in my case because values in that channel might be >=1 I am having nans in the loss function. I did not find any clipping for this part. Have you met such issue previously?

Chuan-10 commented 1 month ago

Hi,

In theory, the uncertainty of each Gaussian ranges from negative infinity to positive infinity. To ensure the rendered values are between 0 and 1, we apply a sigmoid function to the uncertainty before rendering. You can check this here:

During rendering, the uncertainty goes through an alpha-blending process to produce the final 2D image, ensuring pixel values remain between 0 and 1.

Additionally, it is essential to initialize the uncertainty with reasonable values, as shown here:

I haven’t encountered the issue you’re describing before, but I suggest checking if the uncertainty values are processed to be within the 0-1 range before rendering. Hope this helps!