search

hbb1 / 2d-gaussian-splatting

[SIGGRAPH'24] 2D Gaussian Splatting for Geometrically Accurate Radiance Fields
https://surfsplatting.github.io
Other
2.1k stars 144 forks source link

Black shadows around object #120

Open DabblerGISer opened 4 months ago

DabblerGISer commented 4 months ago

Dear all, have you ever be faced with this issue that if you use pics with alpha channel, the output ply file will contain a lot of black shadows(long and flat black gaussians) around the object? 2024-07-22 15-55-44 的屏幕截图

The problem looks like the picture I attached above. How can I solve this problem? I tried to reset the initial color and it doesn't work.

hbb1 commented 4 months ago

I think it is related to these issues https://github.com/hbb1/2d-gaussian-splatting/issues/22 https://github.com/hbb1/2d-gaussian-splatting/issues/42 https://github.com/hbb1/2d-gaussian-splatting/issues/60. It is caused by the introduce of normal consistency loss for RGBA image training. Currently we only use RGB for training. A more effective solution is to use the alpha channel for additional supervision. Or you can use the alpha-channel for masking these regularizations.

DabblerGISer commented 4 months ago

I think it is related to these issues #22 #42 #60. It is caused by the introduce of normal consistency loss for RGBA image training. Currently we only use RGB for training. A more effective solution is to use the alpha channel for additional supervision. Or you can use the alpha-channel for masking these regularizations.

Thank you for your reply!

DabblerGISer commented 4 months ago

it is also possible to remove the black shadows by filter those points whose scale's z-score is beyond threshold(eg. threshold=3, will remove 0.3% of total points). it works efficiently for removing the surrounding shadows.

Zerui-Yu commented 3 months ago

In fact, I found that the point cloud reconstruction of these unbounded scenes was fine, but after visualizing it using SIBR, I could clearly see the errors of alpha, depth and normal. I guess that it may be caused by an anomaly in the size of some Gaussian kernels, and that the results may be improved by modifying the strategy of Densification image image

hbb1 commented 3 months ago

it is also possible to remove the black shadows by filter those points whose scale's z-score is beyond threshold(eg. threshold=3, will remove 0.3% of total points). it works efficiently for removing the surrounding shadows.

Thank you for sharing! what is z-score, like s1/s2?