search

autonomousvision / gaussian-opacity-fields

Gaussian Opacity Fields: Efficient and Compact Surface Reconstruction in Unbounded Scenes
https://niujinshuchong.github.io/gaussian-opacity-fields/
Other
559 stars 26 forks source link

Mistakes in reconstructing Barn and Courthouse of TNT dataset #45

Closed city19992 closed 3 weeks ago

city19992 commented 1 month ago

Thank you for your excellent work! I tried this algorithem on Barn and Courthouse of TNT dataset. While the results show some mistakes.

  1. some noise/floaters are near the camera as follows: 图片5 May I ask why this happened? I guess they may be some gaussians which should be remove in training.

  2. The extracted mesh has floaters in the sky/in the background as follows: 图片6 I need to remove these meshes to get a good results. I wonder why and how to avoid them in training?

The gaussian opacity fields you designed show great performance. May I ask another question that why this work can get good performance in surface reconstruction? Could you please point the reason out in paper or explain it in comments?

niujinshuchong commented 4 weeks ago

Hi, the Courthouse scene is very big and I think sometimes some part of the scene is too close the camera while the gaussian is filtered if the depth is less than 0.2 and in this case, some of the foreground object can't be rendered correct and resulting artifacts as shown in your screenshot. A simple fix is to scale the scene such that most visible part is larger depth than 0.2 or change the cuda code adaptively.

Your results on the Barn scene is reasonable I think. Is that the results from TSDF or marching tetrahedra of opacity field?

The reason why our results are much better than previous work is that our definition of opacity field is consistent with the volume rendering used for rendering RGB images (compared with TSDF or Poisson reconstruction) and also that our regularization help the reconstruction. Please check our paper for more detailed explanations.

city19992 commented 3 weeks ago

Thank you so much for your kind reply. I will further try according to your suggestions. The results on the Barn are from marching tetrahedra of the opacity field. Is there anything in these two methods (TSDF or marching tetrahedra) that will lead to results like this?

niujinshuchong commented 3 weeks ago

Hi, using marching tetrahedra with GOF is much better and it can extract more detailed mesh for both foreground and background.

city19992 commented 3 weeks ago

Thank you so much. But this method reconstruct sky as well. Why and could you please give us some suggestions to remove them?

niujinshuchong commented 3 weeks ago

Hi, we don't have a sky model so the sky will be represented with Gaussians. You need to use sky modeling and semantic segmentation methods to segment the sky region.

city19992 commented 3 weeks ago

Fully understand! Thank you for your reply :)