search

hbb1 / 2d-gaussian-splatting

[SIGGRAPH'24] 2D Gaussian Splatting for Geometrically Accurate Radiance Fields
https://surfsplatting.github.io
Other
1.89k stars 114 forks source link

Black shadows around object #120

Open DabblerGISer opened 1 month ago

DabblerGISer commented 1 month ago

Dear all, have you ever be faced with this issue that if you use pics with alpha channel, the output ply file will contain a lot of black shadows(long and flat black gaussians) around the object? 2024-07-22 15-55-44 的屏幕截图

The problem looks like the picture I attached above. How can I solve this problem? I tried to reset the initial color and it doesn't work.

hbb1 commented 1 month ago

I think it is related to these issues https://github.com/hbb1/2d-gaussian-splatting/issues/22 https://github.com/hbb1/2d-gaussian-splatting/issues/42 https://github.com/hbb1/2d-gaussian-splatting/issues/60. It is caused by the introduce of normal consistency loss for RGBA image training. Currently we only use RGB for training. A more effective solution is to use the alpha channel for additional supervision. Or you can use the alpha-channel for masking these regularizations.

DabblerGISer commented 1 month ago

I think it is related to these issues #22 #42 #60. It is caused by the introduce of normal consistency loss for RGBA image training. Currently we only use RGB for training. A more effective solution is to use the alpha channel for additional supervision. Or you can use the alpha-channel for masking these regularizations.

Thank you for your reply!

DabblerGISer commented 1 month ago

it is also possible to remove the black shadows by filter those points whose scale's z-score is beyond threshold(eg. threshold=3, will remove 0.3% of total points). it works efficiently for removing the surrounding shadows.

Zerui-Yu commented 1 month ago

In fact, I found that the point cloud reconstruction of these unbounded scenes was fine, but after visualizing it using SIBR, I could clearly see the errors of alpha, depth and normal. I guess that it may be caused by an anomaly in the size of some Gaussian kernels, and that the results may be improved by modifying the strategy of Densification image image

hbb1 commented 1 month ago

it is also possible to remove the black shadows by filter those points whose scale's z-score is beyond threshold(eg. threshold=3, will remove 0.3% of total points). it works efficiently for removing the surrounding shadows.

Thank you for sharing! what is z-score, like s1/s2?

HuangWenH commented 4 weeks ago

Is there any solution? I encountered the same problem. If I remove the normal loss, the mesh extraction effect will be very poor. When I adjusted the normal loss start time to 15k, it did not solve the @Zerui-Yu @hbb1 b226704e325b02fc68e9af95fceaf80 20240814204304